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1.  Introduction 


1.1  Statement  of  the  Problem 

Several  streams  of  research  are  under  way  to  delineate  the  degree  and  manner  in  which  robotic 
controllers  and  displays  can  be  scaled  to  the  needs  of  Army  Soldiers  (Barnes,  Knapp,  Tillman, 
Walters,  &  Velicki,  2000;  Barnes,  Everett,  &  Rudakevych,  2005;  Chen,  Haas,  &  Barnes,  2007; 
Chen,  Haas,  Pillalamarri,  &  Jacobson,  2006;  Renfro,  Merlo,  Duley,  Gilson,  &  Hancock,  2007; 
Stafford,  Jingjing,  Merlo,  &  Hancock,  2007;  Stafford,  Hancock,  Graham,  &  Merlo,  2007).  This 
experiment  is  the  second  in  a  series  of  experiments  designed  to  investigate  current  and  future 
options  for  scaling  robotic  controls  and  displays  specifically  for  use  by  dismounted  Soldiers. 

The  term  “scalability”  encompasses  the  various  ways  in  which  devices  can  be  made  more  opera¬ 
tionally  effective,  not  only  in  terms  of  size  and  weight  but  also  in  terms  of  more  intuitive  display 
of  information,  enhancement  of  training  transfer  among  other  Army  controls  and  displays,  and 
minimization  of  information  overload.  For  example,  one  operational  definition  of  scalability  is 
“The  transmission  of  critical  information  to  the  Soldier  tailored  for  each  level  of  combat  to  ensure 
mission  success  while  maximizing  survivability  by  minimizing  equipment  requirements;  mini¬ 
mizing  multitasking  workload,  maximizing  situation  understanding;  and  maximizing  aerial  and 
ground  robotic  mission  effectiveness”  (Merlo,  2006).  A  similar  definition  is  “The  tailored 
reception  and  transmission  of  mission-essential  information  at  the  appropriate  level  for  the 
Soldier,  to  ensure  mission  success  while  maximizing  the  survivability  and  lethality  through  the 
synergistic  interaction  of  equipment  requirements,  appropriate  cognitive  workload,  situation 
awareness  and  understanding  for  oneself  and  others  connectivity  of  distributed  intelligent  agents” 
(Barnes,  2006). 

In  our  stream  of  research,  we  focus  on  size,  weight,  and  ease  of  use  for  dismounted  Soldier 
applications.  Soldiers  operate  in  a  large  range  of  environments,  from  the  relatively  stable  and 
spacious  environment  of  a  tactical  operations  center  (TOC)  to  the  cramped  environment  of  a 
vehicle  to  the  rugged  and  challenging  environment  of  the  dismounted  Soldier.  All  these  en¬ 
vironments  have  an  impact  on  the  effective  size  and  configuration  of  the  robotic  interface.  A 
dismounted  Soldier  cannot  carry  the  relatively  large  controller  that  can  be  used  in  a  TOC.  Small 
input  devices  are  quite  different  from  larger  ones  and  can  potentially  impact  the  complexity  of 
operation,  the  speed  of  operation,  and  the  accuracy  of  input.  Designing  for  the  “optimum”  input 
device  size  may  seem  to  be  a  good  idea  because  the  presentation  of  the  interface  may  seem  to  be 
controlled.  However,  if  an  input  device  is  designed  that  is  only  practical  in  one  environment,  it 
may  be  completely  unusable  in  another. 
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Certainly,  smaller  devices  that  weigh  less  are  usually  preferred  by  dismounted  Soldiers.  The 
question  is  how  best  do  we  accomplish  controller  size  reduction?  Tele-operation  of  robotic 
vehicles  is  a  difficult  task  because  robots  are  complex  machines,  typically  consisting  of  a  large 
number  of  components.  Many  of  the  robotic  components  provide  feedback  to  the  operator  and 
require  input  from  the  operator.  Typically,  the  controller  consists  of  at  least  two  joysticks  or 
comparable  controls  for  robotic  navigation  and  camera  control.  In  addition,  there  are  many 
buttons  and  switches  and  one  or  more  video  screens.  Space  to  incorporate  controls  on  small 
controllers  is  very  limited. 

Miniaturizing  individual  controls  is  one  option  for  reducing  the  size  of  controllers,  but  Soldiers 
have  to  be  able  to  operate  controllers  while  wearing  gloves  without  accidentally  activating  adjacent 
controls.  One  benefit  of  the  miniaturization  approach  is  that  if  we  simply  make  existing  control¬ 
lers  smaller,  the  similarity  of  controls  should  enhance  transfer  of  training.  However,  how  small 
can  we  go  before  the  buttons  and  spacing  are  too  difficult  for  operational  performance?  Another 
option  is  to  reduce  the  number  of  individual  controls  on  controller  devices  by  incorporating 
multifunction  controls  (controls  that  can  perform  different  functions,  depending  on  the  current 
mode  of  operation).  For  example,  a  driving  joystick  can  become  a  camera  pan-and-tilt  controller 
when  the  system  is  placed  in  another  mode  of  operation.  However,  multifunction  controls  may 
increase  control  activation  time  and  cognitive  complexity.  The  end  result  could  be  that  a  controller 
with  multifunction  controls  may  require  many  hours  of  training  before  a  user  is  competent  and 
comfortable  enough  to  use  the  system  during  a  real  mission.  Other  creative  approaches  to  control¬ 
ler  size  reduction  that  have  been  developed  are  sketch  interfaces  (Skubic,  Bailey  &  Chronis,  2003; 
Setalaphruk,  Ueno,  Kume,  &  Kono,  2003),  voice  recognition  and  synthesis  systems  (Chen,  Haas, 
Pillalamarri,  &  Jacobson,  2006),  and  hands-free  systems  (Veronka  &  Nestor,  2001),  but  these 
novel  controls  often  present  problems  of  their  own  and  they  are  still  being  refined. 

In  the  meantime,  there  is  an  immediate  requirement  for  dismounted  Soldier  robotic  tele-operation 
controllers  and  not  much  empirical  data  about  the  impact  of  size  reduction.  In  this  experiment,  we 
chose  three  different  types  of  controllers  and  demonstrated  their  potential  for  use  by  a  dismounted 
Soldier  to  control  a  small  robotic  platform  and  its  sensor  payload.  One  of  the  controllers  was  the 
default  controller  for  the  robot  used  in  this  study.  We  also  included  two  controllers  that  reflected 
near-term  options  for  scaling  controllers  for  the  dismounted  Soldier.  All  controllers  were  com¬ 
pared  on  the  same  set  of  measures.  Pre-test  experimentation  was  performed  to  ensure  that  the 
functional  mapping  of  each  of  the  controllers  was  effective  as  possible.  The  trade-offs  between 
interface  size  and  weight  with  these  controllers  versus  input  speed,  accuracy,  training  time,  and 
cognitive  load  were  examined. 

1.2  Overview  of  Experiment 

This  study  was  an  investigation  of  the  effect  of  controller  size  scalability  on  robotic  control.  It 
took  place  at  Fort  Benning,  Georgia,  with  12  Soldiers  from  the  Officers’  Candidate  School 
(OCS)  and  three  Soldiers  from  Headquarters  Company  1st  Battalion  11th  Infantry  Regiment  as 
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participants.  Before  any  training,  an  intuitiveness  evaluation  was  conducted  on  each  controller. 
After  training  in  the  operation  of  the  IRobot1  PackBot1  Robot  system,  each  Soldier  completed  a 
driving  course  using  three  different  controller  types.  The  terrain  and  hazards  were  counter¬ 
balanced  along  with  the  controller  type  to  control  for  the  effect  of  learning.  Workload  for  each 
controller  was  measured  by  having  the  Soldiers  complete  the  National  Aeronautics  and  Space 
Administration  (NASA)  Task  Load  Index  (TLX)  survey  after  they  completed  the  driving  course. 
Controller  type  and  usability  were  evaluated,  based  on  objective  performance  data,  data  collector 
observations,  and  Soldier  questionnaires. 

1.3  Objectives 

There  were  two  objectives  in  this  experiment.  The  primary  objective  was  to  determine  whether 
differences  among  the  controller  design  approaches  had  any  effect  on  the  operation  of  a  small 
robot.  The  second  objective  was  to  conduct  an  intuition  test  of  the  controller  design  and  function 
mapping  of  each  controller. 


2.  Method 


2.1  Participants 

Twelve  Soldiers  from  the  OCS  and  three  Soldiers  from  Headquarters  Company  1st  Battalion  1 1th 
Infantry  Regiment  participated  in  the  study. 

2.1.1  Pre-test  Orientation  and  Volunteer  Agreement 

The  Soldiers  were  given  an  orientation  about  the  purpose  of  the  study  and  their  participation. 

They  were  briefed  about  the  objectives  and  procedures,  as  well  as  the  robot.  They  were  also  told 
how  the  results  would  be  used  and  the  benefits  the  military  could  expect  from  this  investigation. 
Any  questions  the  subjects  had  regarding  the  study  were  answered.  It  was  made  clear  that  Soldier 
participation  in  the  experimentation  was  voluntary.  The  volunteer  agreement  affidavit  was  ex¬ 
plained  and  its  contents  verbally  presented.  The  Soldiers  were  given  the  volunteer  agreement 
affidavit  to  read  and  sign  if  they  decided  to  volunteer. 

2.1.2  Medical  Review  and  Screening 

The  Soldiers  were  given  a  medical  status  form  to  determine  if  any  of  them  had  a  medical  profile 
or  history  that  would  jeopardize  them  if  they  participated  in  the  study. 


1  i  Robot  and  PackBot  are  registered  trademarks  of  iRobot  Corporation,  Burlington,  Massachusetts. 
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2.2  Instruments  and  Apparatus 


2.2.1  IRobot  PackBot  Explorer  Robot 

The  IRobot  PackBot  Explorer  Robot  (figure  1)  is  a  portable  small  unmanned  ground  vehicle 
(SUGV)  reconnaissance  and  tactical  robot  that  can  enter  and  secure  areas  that  are  inaccessible  or 
too  dangerous  for  humans.  The  IRobot  PackBot  Explorer  Robot  payload  has  a  rotating  pan-and- 
tilt  head  equipped  with  multiple  cameras. 

2.2.2  Robotic  Vehicle  Controllers 

The  three  controller  types  used  during  this  experiment  are  listed  in  table  1 .  Controller  A  was  the 
largest  of  the  three  controllers,  and  each  control  manipulation  had  a  single  function.  Controllers  A 
and  B  had  the  same  number  of  single-function  controls;  however,  controller  B’s  controls  were 
placed  in  a  different  configuration  and  were  smaller  than  controller  A’s.  Controller  C  attached  to 
the  bottom  rail  of  the  M4  in  the  same  fashion  as  a  forward  hand  grip.  Controller  C  had  the  fewest 
controls,  and  the  controls  had  multi-functionality.  Controllers  A  and  B  were  designed  to  be 
operated  with  both  hands,  while  controller  C  was  designed  for  single-handed  operation.  Pictures 
of  the  three  controllers  are  shown  in  figures  2  through  4. 


Figure  1.  IRobot  PackBot  explorer  robot. 


Tablet.  Controller  data. 


Controller  Label 

Controller  Name 

No.  of  Possible 
Control 
Manipulations 

Controller  Weight 
(ib) 

Controller 
Dimensions 
(LxWxH  in.) 

A 

Legacy  Controller 

30 

1.40 

8.5x4.25x4.25 

B 

Logitech  Controller 

26 

.56 

6x4x2.38 

c 

Q3D  Controller 

11 

.65 

6x2.63x7 

4 


DOCU  for  Explorer: 

Mappings 
(Legacy  Controller) 


Left  Stick  =  Drive 


▲  =  Forward 
T  =  Backward 
◄  =  Left 
►  =  Right 


Right  Stick  =  Camera 
A  =  Tilt  Up 
▼  =  Tilt  Down 
◄  =  Pan  Left 
►  =  Pan  Right 


Figure  2.  Controller  A. 


DOCU  for  Explorer: 
Mappings 


Figure  3.  Controller  B. 


2Logitech  is  a  registered  trademark  of  Logitech,  Inc. 
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Figure  4.  Controller  C. 


2.2.3  Control  Intuitiveness  Test 

The  control  intuitiveness  test  performed  in  this  experiment  was  roughly  based  on  an  icon 
intuitiveness  study  done  by  Nielsen  and  Sano  (1994),  which  was  performed  during  the  usability 
evaluation  of  Sun  Microsystems’  internal  web.  Nielsen  and  Sano  presented  images  to  users  who 
were  then  asked  to  indicate  what  functionality  they  thought  the  icons  represented.  In  the  present 
study,  we  presented  a  task  (i.e.,  raise  the  arm  of  the  sensor  head,  raise  the  flippers,  drive  the  robot 
forward,  brake,  pan  the  camera  to  the  left)  to  the  Soldiers  who  had  no  prior  training  or  use  of  the 
controller  and  then  asked  them  to  perform  the  task.  The  administrator  documented  the  number  of 
correct  and  incorrect  attempts  to  perform  the  task  and  noted  the  specific  controls  that  were 
incorrectly  activated. 

2.2.4  Robotic  Course 

The  experiment  was  conducted  on  a  driving  course  approximately  200  meters  long  (figure  5).  The 
driving  lane  on  the  course  was  approximately  1  meter  wide  and  clearly  marked  with  white  engineer 
tape  on  the  left  and  right  sides.  Along  the  course  were  seven  stations  where  the  operator  conducted 
a  specified  maneuver  or  reconnaissance  task.  The  seven  stations  were  stair  ascend/descend;  win¬ 
dow  reconnaissance;  target  identification/tracking;  hill  climb;  bunker  reconnaissance;  tunnel  recon¬ 
naissance;  and  zigzag  negotiation.  Soldiers  tele-operating  the  IRobot  PackBot  were  situated  at  a 
remote  stationary  position  inside  a  tent.  The  tent  kept  the  operator  and  the  controller  equipment  out 
of  the  elements.  A  description  of  each  station  and  instructions  for  executing  them  follow: 
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•  Station  1.  Stairs  (Maneuver).  The  stairs  are  made  of  wood.  Five  steps  lead  up  to  a 
platform  and  five  steps  lead  down.  In  order  to  ascend  and  descend  the  stairs,  the  operator 
must  correctly  position  the  robot  flippers  to  obtain  the  appropriate  angle  and  control  the 
robot’s  speed  throughout  the  maneuver. 

•  Station  2.  Window  (Maneuver/Reconnaissance).  The  operator’s  task  at  this  station  is  to 
locate  and  identify  the  simulated  improvised  explosive  device  (IED)  hidden  behind  the 
urban  wall.  Upon  reaching  the  wall,  the  operator  must  back  the  robot  into  position  below 
the  window,  extend  the  sensor  head  arm,  and  maneuver  the  sensor  head  in  order  to  see 
through  the  window  opening.  The  operator  locates  and  identifies  to  the  data  collector  a 
pipe  bomb,  ammunition  can  bomb,  or  soda  can  bomb. 

•  Station  3.  Target  Tracking  (Maneuver/Reconnaissance).  The  operator  positions  the  robot 
at  the  row  of  sandbags  and  locates  the  human  target.  The  operator  zooms  in  on  the  human 
target  and  identifies  the  object  he  is  holding  as  a  sledge  hammer,  M4  carbine,  shovel,  or 
box.  The  human  target  is  then  cued  by  the  data  collector  to  start  walking  perpendicular  to 
the  robot.  At  set  intervals  along  the  route,  the  human  target  discards  the  object  and 
replaces  it  with  a  different  object.  The  operator  is  required  to  track  the  target  and  report 
to  the  data  collector  what  the  target  is  carrying  at  all  times  throughout  the  exercise. 

•  Station  4.  Hill  (Maneuver).  The  hill  is  approximately  10  m  long,  1  m  wide,  and  3  m  tall 
with  a  30-degree  sloped  incline  and  decline.  The  operator  must  control  the  robot’s 
direction  and  speed  throughout  the  maneuver. 

•  Station  5.  Bunker  Reconnaissance  (Foxhole)  (Maneuver/Reconnaissance).  The  operator’s 
task  at  this  station  is  to  locate  and  identify  the  simulated  IED  marked  with  an  alphanumeric 
label  hidden  in  a  partially  covered  foxhole.  Upon  reaching  the  foxhole,  the  operator  must 
back  the  robot  into  position,  extend  the  sensor  head  arm,  and  maneuver  the  sensor  head  in 
order  to  see  through  the  foxhole  opening.  After  the  operator  locates  the  IED,  s/he  must  use 
the  camera  zoom  feature  and  report  the  alphanumeric  label  to  the  data  collector. 

•  Station  6.  Tunnel  (Maneuver).  The  tunnel  is  a  culvert  pipe  6  m  long  by  1  m  in  diameter 
and  made  of  corrugated  steel.  The  operator  aligns  the  robot  with  the  tunnel  opening  and 
maneuvers  the  robot  through  the  pipe. 

•  Station  7.  Zigzag  (Maneuver).  The  zigzag  is  1.6  m  tall,  14  m  long,  and  approximately  1  m 
wide.  It  consists  of  three  turns  (approximately  90  degrees  each)  within  the  lane.  The  frame¬ 
work  is  constructed  of  wood  with  mesh  wire  installed  between  the  two  lanes  and  on  the 
outside  framework  of  each  lane.  The  operator  maneuvers  the  robot  through  the  zigzag, 
controlling  both  speed  and  direction. 
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Figure  5.  Robotic  driving  course. 
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2.2.5  NASA  TLX 


The  NASA  TLX  requires  the  user  to  rate  the  workload  of  a  device  on  a  number  of  different  scales 
and  to  assign  an  importance  weight  to  each  scale.  The  scores  on  the  workload  scales  (mental, 
physical,  temporal,  performance,  effort,  and  frustration)  can  be  combined  in  an  overall  workload 
score  (Hart  &  Staveland,  1988). 

2.2.6  Questionnaires 

The  questionnaires  were  designed  to  elicit  Soldiers’  opinions  about  their  performance  and 
experiences  with  each  of  the  controller  systems.  The  questionnaires  asked  the  Soldiers  to  rate 
the  devices  on  a  7-point  Likert  scale  ranging  from  “extremely  bad/difficult”  to  “extremely 
good/easy”.  Questionnaires  were  administered  to  each  Soldier  at  the  end  of  each  iteration  and  at 
the  end  of  the  experiment.  Questionnaires  were  also  used  to  gather  information  concerning  the 
Soldiers’  demographic  data,  robotic  experience,  and  physical  characteristics  that  might  affect  their 
ability  to  operate  the  robot. 

2.3  Procedures  and  Methodology 
2.3.1  Soldier  Orientation 

The  experiment  Soldiers  reported  in  groups  of  three  for  one  day  each,  0800  to  1800  daily.  They 
then  received  a  roster  number,  which  was  used  to  identify  them  throughout  the  evaluation.  After 
their  pre-test  orientation,  the  volunteer  agreement  affidavit  was  explained  and  its  contents  verbally 
presented.  Next,  the  Soldiers  were  given  the  volunteer  agreement  affidavit  to  read  and  sign  if  they 
decided  to  volunteer.  The  volunteers  were  then  asked  to  complete  the  medical  review  and  screen¬ 
ing  form.  Demographic  data,  as  well  as  data  concerning  their  Army  and  robotic  experience  and 
physical  characteristics,  were  also  taken  for  each  Soldier.  Next,  Soldiers  received  an  orientation 
of  the  robotics  driving  course.  During  the  orientation,  trainers  demonstrated  the  tasks  to  be 
performed  at  each  station.  The  intuitive  test  and  the  training  course  were  also  explained  and  any 
questions  the  Soldiers  had  concerning  the  experiment  were  answered. 

Soldiers  completed  three  iterations  of  the  four  stations  (one  with  each  controller  type)  according 
to  the  matrix  in  table  2.  They  started  at  station  1  (the  intuitiveness  exercise),  then  moved  to 
station  2  (the  training  course),  then  moved  to  station  3  (the  robotic  course),  and  finally  moved 
to  station  4  (the  questionnaire  station).  After  the  third  iteration,  they  completed  an  end  of  experi¬ 
ment  questionnaire. 
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Table  2.  Order  of  treatments. 


Roster  (Soldier) 

Iteration 

1 

2 

3 

1 

A 

B 

C 

2 

B 

C 

A 

3 

c 

A 

B 

4 

c 

B 

A 

5 

B 

A 

c 

6 

A 

c 

B 

7 

A 

B 

C 

8 

B 

c 

A 

9 

c 

A 

B 

10 

C 

B 

A 

11 

B 

A 

C 

12 

A 

c 

B 

13 

A 

B 

c 

14 

B 

c 

A 

15 

C 

A 

B 

2.3.2  Control  Intuitiveness  Station 

At  the  intuitiveness  station,  Soldiers  were  given  a  controller  with  all  labels  covered  or  removed 
and  were  asked  to  activate  the  control  that  they  thought  would  most  likely  cause  the  robot  to 
perform  a  specified  task.  Soldiers  were  then  asked  why  they  selected  the  control.  Five  tasks 
were  selected  for  the  test:  raise  the  arm  of  the  sensor  head,  raise  the  flippers,  drive  the  robot 
forward,  brake,  and  pan  the  camera  to  the  left.  Soldiers  were  given  three  attempts  to  select  the 
correct  control  and  were  asked  after  each  selection  why  they  selected  it.  If  a  Soldier  selected  the 
correct  control,  s/he  moved  to  the  next  task.  The  administrator  documented  the  number  of  incor¬ 
rect  attempts  to  perform  the  task,  noted  the  specific  controls  that  were  incorrectly  activated,  and 
recorded  the  Soldier’s  feedback  concerning  why  each  control  input  was  chosen.  After  the  Soldiers 
completed  the  tasks  with  all  three  controllers,  they  were  asked  to  complete  a  questionnaire 
designed  to  elicit  information  about  the  intuitiveness  of  each  controller’s  design. 

2.3.3  Training  Station 

A  representative  from  iRobot  trained  the  Soldiers  to  use  the  IRobot  PackBot.  Soldiers  practiced 
tele-operating  the  robot  on  the  actual  course  used  during  the  experiment.  They  were  trained  on 
each  controller  just  before  executing  the  robotic  course  with  that  controller.  The  course  required 
driving,  obstacle  negotiation,  sensor  head  manipulation,  target  identification,  and  moving  target 
tracking.  It  also  involved  training  in  the  graphical  user  interface  (GUI),  use  of  the  video  cameras, 
and  use  of  the  robotic  flippers  to  negotiate  obstacles  such  as  stairs  and  low  walls.  Soldiers  were 
trained  on  each  controller  for  30  minutes.  During  pilot  testing,  it  was  found  that  30  minutes  of 
hands-on  training  with  each  controller  was  adequate  for  Soldiers  to  become  proficient  in  per¬ 
forming  the  various  tasks  required  to  complete  the  driving  course.  Upon  completion  of  the 
training,  the  Soldiers  negotiated  the  robotic  driving  course. 
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2.3.4  Robotic  Driving  Course 

Soldiers  negotiated  the  robotic  driving  course  by  tele-operating  the  robot  using  the  three  controller 
types.  The  course  required  driving,  obstacle  negotiation,  sensor  head  manipulation,  target  identi¬ 
fication,  and  moving  target  tracking. 

The  robotics  course  was  approximately  200  m  long  and  had  seven  stations  along  its  route  (see 
table  3).  The  Soldiers  were  told  to  maneuver  the  robot  along  the  course  marked  with  engineer  tape 
and  to  conduct  a  reconnaissance  at  key  locations  along  the  route.  At  each  station,  the  operator  was 
instructed  to  perform  a  specified  maneuver  or  reconnaissance  task.  A  data  collector  following  the 
robot  recorded  the  time  required  for  the  Soldier  to  complete  each  station,  driving  errors,  and  the 
number  of  times  the  robot  was  driven  off  course.  A  driving  error  was  defined  as  running  into 
objects  on  the  course  or  failing  to  properly  position  the  robot  to  perform  the  task  required  at  a 
station.  Off  course  was  defined  as  major  off  course  or  minor  off  course.  A  major  off  course  was 
recorded  if  the  robot  went  completely  outside  the  engineer  tape  and  a  minor  off  course  was  re¬ 
corded  when  only  a  portion  of  the  robot  crossed  the  tape.  An  additional  data  collector  was  co¬ 
located  with  the  Soldier  to  observe  and  record  the  Soldier’s  responses  and  to  record  overall  course 
completion  times. 


Table  3.  Robotic  course  stations. 


Station 

Description 

Task 

Station  1 

Stair  Ascend/Descend 

Maneuver 

Station  2 

Window  Target 

Maneuver/Reconnaissance 

Station  3 

Area  Scan/moving  target 

Maneuver/Reconnaissance 

Station  4 

Hill 

Maneuver 

Station  5 

Bunker  Reconnaissance  (Foxhole) 

Maneuver/Reconnaissance 

Station  6 

Tunnel 

Maneuver 

Station  7 

ZigZag 

Maneuver 

Upon  completion  of  the  course,  the  Soldiers  were  given  a  questionnaire  designed  to  assess  their 
performance  and  experiences  with  each  of  the  control  systems  and  to  assess  their  perception  of 
the  training  adequacy.  Questions  about  the  amount  of  practice  time  given,  the  level  of  detail 
presented,  and  the  adequacy  of  training  aids  were  included. 

2.3.5  End  of  Iteration  Questionnaire  Administration  Station 

End  of  iteration  questionnaires  and  the  NASA-TLX  were  administered  to  each  Soldier  at  the  end 
of  each  iteration.  After  completing  the  course  with  each  of  the  three  controller  types,  the  Soldiers 
completed  an  end  of  experiment  questionnaire  that  compared  each  of  the  controller  types  for  a 
number  of  characteristics.  They  also  completed  questionnaires  concerning  the  information 
requirements  for  tele-operating  the  robot. 
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2.4  Experimental  Design 

This  experiment  used  a  within- subject  design,  with  treatments  counterbalanced  to  control  for 
sequence  effects. 

2.4.1  Independent  Variable 

•  Controller  type 

2.4.2  Dependent  Variables 

•  NASA  TLX  workload  scores, 

•  Robotic  course  completion  time, 

•  Time  to  complete  each  station  on  the  course, 

•  The  number  of  major  times  the  driver  went  off  course, 

•  The  number  of  minor  times  the  driver  went  off  course, 

•  The  number  of  driving  errors  on  the  course, 

•  Number  and  type  of  errors  committed  during  each  task  of  the  intuition  test, 

•  Data  collector  comments,  and 

•  Questionnaire  responses. 

2.5  Data  Analysis 

All  objective  data  were  analyzed  with  a  repeated  measures  analysis  of  variance  (ANOVA). 
Follow-on  pair-wise  comparisons  were  done  with  Holm’s  Bonferroni  procedures  to  control  for 
family-wise  error  rates.  Partial  r\2  (r|2p),  an  index  of  effect  size,  was  computed  for  each  ANOVA. 
Iteration  effects  were  controlled  through  the  counterbalanced  order  of  the  experimental  design. 
Soldier  questionnaire  data  were  analyzed  with  the  use  of  descriptive  statistics  on  the  subjective 
ratings. 


3.  Results 


3.1  Demographics 

The  rank  of  the  enlisted  Soldiers  ranged  from  E-3  to  E-7,  with  one  chief  warrant  officer  in  the 
rank  of  CW3.  Their  average  time  in  service  was  69  months.  Two  of  the  Soldiers  wrote  with 
their  left  hand  and  all  15  Soldiers  fired  their  weapons  right  handed.  Nine  of  the  Soldiers  had 
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driven  a  remote  controlled  car,  but  none  had  ever  operated  a  robotic  vehicle.  All  but  three  of  the 
Soldiers  played  video  games  on  a  regular  basis.  Detailed  results  are  presented  in  appendix  A. 

3.2  Control  Intuitiveness 

The  results  from  the  intuitiveness  test  were  weighted  based  on  the  number  of  attempts  to  find  the 
correct  control.  Three  points  were  given  for  the  first  selection,  two  for  the  second,  and  one  for 
the  third.  Therefore,  the  higher  weights  represent  the  most  intuitive  control  mapping  and  the 
lower  weights  represent  the  least  intuitive. 

The  following  three  figures  show  each  controller  with  the  controls  labeled  with  numbers  used  to 
identify  which  control  the  Soldiers  selected  during  the  intuitiveness  test.  The  following  three 
tables  show  the  tasks,  the  control  with  the  highest  intuitiveness  weight,  and  the  weight  given  for 
the  control  currently  mapped  to  perform  the  task.  The  complete  list  of  responses  (including  those 
that  did  not  receive  the  highest  weightings)  are  presented  in  appendix  B. 

3.2.1  Controller  A 

The  controls  on  controller  A  labeled  2  through  17  are  actuated  by  buttons;  1A  through  IF  and  18A 
through  18F  are  actuated  by  toggle  manipulations  (figure  6).  Results  for  controller  A  (table  4) 
show  that  the  current  controller  mappings  for  raising  the  arm  of  the  sensor  head  (18E)  and  raising 
the  flippers  (IF)  were  not  selected  as  the  most  intuitive  control  manipulations  to  perform  these 
tasks.  Soldiers  justified  their  selections,  based  on  the  assumption  that  the  toggles  were  mapped  for 
important  functions  such  as  driving  and  camera  manipulations.  The  current  mapping  for  driving 
the  robot  forward,  applying  brakes,  and  panning  the  camera  left  were  weighted  as  the  most 
intuitive  control  manipulations  for  these  tasks.  Detailed  results  are  presented  in  appendix  B. 


Figure  6.  Controller  A  with  labels. 
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Table  4.  Controller  A  weighted  scores. 


Task 

Control  Selected 
(figure  6) 

Weighted  Score 

1 .  Raise  the  arm  of  the  sensor  head. 

2 

13 

18A 

13 

18E* 

2* 

2.  Raise  the  flippers 

2 

11 

IF* 

3* 

3.  Drive  the  robot  forward. 

1A* 

37* 

4.  Brake 

14* 

12* 

5.  Pan  the  camera  left 

18C* 

17* 

*  =  Currently  mapped  control 


3.2.2  Controller  B 

The  controls  on  controller  B  (figure  7)  labeled  1  through  8  and  1 1  through  1 8  are  actuated  by 
buttons;  9A  through  9D  and  10A  through  10D  are  actuated  by  toggle  manipulations.  Results  for 
controller  B  (table  5)  show  that  the  current  controller  mappings  for  raising  the  arm  of  the  sensor 
head  (2)  and  applying  brakes  (7)  were  not  selected  as  the  most  intuitive  control  manipulations  to 
perform  these  tasks.  Soldiers  based  their  selection  of  10A  for  raising  the  sensor  head  on  the 
assumption  that  the  toggles  should  control  the  camera  and  14  for  braking  based  on  the  assumption 
that  the  directional  pad  should  control  the  robot’s  speed.  The  current  mappings  for  driving  the 
robot  forward,  raising  the  flippers,  and  panning  the  camera  left  were  weighted  as  the  most  intuitive 
control  manipulations  for  these  tasks. 


Figure  7.  Controller  B  with  labels. 
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Table  5.  Controller  B  weighted  scores. 


Task 

Control  Selected 
(figure  7) 

Weighted  Score 

1 .  Raise  the  arm  of  the  sensor  head. 

10A 

16 

2* 

12* 

17* 

22* 

2.  Raise  the  flippers 

9A* 

36* 

14 

18 

3.  Drive  the  robot  forward. 

7* 

5* 

4.  Brake 

IOC* 

21* 

5.  Pan  the  camera  left 

10A 

16 

*  =  Currently  mapped  control 


3.2.3  Controller  C 

The  controls  labeled  2  through  5  on  controller  C  (figure  8)  are  buttons;  1A  through  IE  are 
actuated  by  a  toggle,  and  6 A  and  B  are  actuated  when  the  hand  grip  on  the  control  is  twisted. 
Results  for  controller  C  (table  6)  show  that  the  current  controller  mapping  for  raising  the  arm  of 
the  sensor  head  (ID)  was  not  selected  as  the  most  intuitive  control  manipulation  to  perform  the 
task.  Soldiers  stated  that  raising  and  lowering  the  arm  of  the  sensor  head  should  be  accomplished 
by  pushing  up  or  down  on  the  toggle.  The  current  mappings  for  raising  the  flippers,  driving  the 
robot  forward,  applying  brakes,  and  panning  the  camera  left  were  weighted  as  the  most  intuitive 
control  manipulations  for  these  tasks. 


Figure  8.  Controller  C  with  labels. 
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Table  6.  Controller  C  weighted  scores. 


Task 

Control  Selected 
(figure  8) 

Weighted  Score 

1 .  Raise  the  arm  of  the  sensor  head. 

1A 

34 

ID* 

2* 

1A* 

21* 

2.  Raise  the  flippers 

1A* 

39* 

5* 

18* 

3.  Drive  the  robot  forward. 

1C* 

43* 

4.  Brake 

1A 

34 

5.  Pan  the  camera  left 

ID* 

2* 

*  =  Currently  mapped  control 


3.2.4  Summary 

Table  7  and  figure  9  show  the  proportion  of  correct  responses  (choice  of  the  currently  mapped 
control)  to  each  command  for  the  three  controllers.  A  correct  response  is  defined  as  the  response 
being  correct  on  the  first,  second,  or  third  choice.  The  differences  in  proportion  of  total  correct 
responses  between  the  three  controllers  is  statistically  significant:  %  =  9.90,  df  =  2,p  <  .01.  One 
reason  for  the  higher  proportion  of  correct  responses  with  the  C  controller  is  that  there  are  only 
1 1  possible  responses  with  the  C  device,  as  opposed  to  30  possible  responses  with  the  A 
controller  and  26  possible  responses  with  the  B  controller. 


Table  7.  Proportion  of  correct  responses. 


Task 

Controller  (percent) 

A 

B 

c 

1 .  Raise  the  arm  of  the  sensor  head 

3 

16 

3 

2.  Raise  the  flippers 

3 

26 

29 

3.  Drive  the  robot  forward. 

74 

52 

100 

4.  Brake 

21 

7 

25 

5.  Pan  the  camera  left 

28 

30 

82 

16 


Figure  9.  Proportion  of  correct  responses. 


3.3  Training 

The  Soldiers’  overall  ratings  of  the  training  course  were  between  good  and  extremely  good  for 
all  three  controllers.  Most  test  Soldiers  rated  the  C  controller  as  more  difficult  to  learn  than  the 
A  and  B  controllers  primarily  because  they  had  to  learn  the  different  functions  (modes)  for  each 
control.  Controlling  the  camera  and  adjusting  to  the  controllers’  sensitivities  were  identified  as 
tasks  that  were  difficult  to  learn  for  all  three  controllers.  Soldiers  commented  that  the  30  minutes 
allowed  to  train  with  each  controller  was  adequate  time  for  learning  to  negotiate  the  course. 
Detailed  training  questionnaire  results  are  presented  in  appendix  A. 

3.4  Robotic  Course 

Table  8  and  figure  10  show  the  mean  course  completion  times  for  the  three  controllers.  The  error 
bars  in  the  graph  represent  95%  confidence  intervals.  A  repeated  measures  ANOVA  indicated  that 
there  was  a  significant  difference  among  the  means:  F(2,24)  =  7.07,/?  =  .004,  r\  p  =  .371.  Follow- 
on  pair-wise  comparisons  were  conducted  with  Holm’s  Bonferroni  sequential  correction  for 
family-wise  error  rates.  As  shown  in  table  9,  the  average  course  completion  time  for  controller  C 
was  significantly  slower  than  the  average  times  for  controllers  A  and  B.  The  time  required  to 
switch  function  modes  and  the  limited  ability  to  perform  simultaneous  tasks  when  Soldiers  were 
tele-operating  the  robot  using  controller  C  were  identified  as  the  primary  reasons  for  its  slower 
completion  times.  Performing  some  tasks  simultaneously  with  controller  C,  such  as  driving  the 
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robot  forward  while  raising  the  sensor  arm,  is  not  possible  because  Soldiers  have  to  switch  modes 
of  the  multifunction  controls. 


Table  8.  Mean  course  completion  times. 


Controller 

Mean 

SD 

A 

16:17 

3:44 

B 

16:07 

5:49 

C 

20:35 

3:37 

18 


Table  9.  Follow-on  comparisons,  course  completion  times. 


Pair 

T 

df 

obtained  p 

required  p 

A  vs.  B 

0.22 

14 

0.827 

0.05 

A  vs.  C 

-3.84 

12 

0.002* 

0.0167 

B  vs.  C 

-2.84 

12 

0.015* 

0.025 

* p  <  .05,  2-tailed 


Table  10  shows  the  mean  number  of  major  off-course  errors.  There  were  too  few  major  errors  to 
conduct  a  repeated  measures  ANOVA. 


Table  10.  Mean  major  off-course  errors. 


Controller 

Mean 

SD 

A 

2.00 

0 

B 

1.67 

0.52 

C 

1.67 

0.58 

The  mean  numbers  of  minor  off-course  errors  for  the  three  controllers  are  shown  in  table  1 1 . 
There  was  no  statistically  significant  difference  among  the  means:  F(2, 14)  <  1.00,/?  =  .639, 
r|2p  =  .062. 


Table  11.  Mean  minor  off-course  errors. 


Controller 

Mean 

SD 

A 

4.13 

4.64 

B 

4.88 

2.70 

C 

3.38 

1.51 

The  average  numbers  of  driving  errors  between  stations  for  each  controller  are  shown  in  table  12. 
There  was  no  statistically  significant  difference  among  the  means:  F(2, 16)  =  1.17,/?  =  .336, 
ri2p  =  .128. 


Table  12.  Mean  driving  errors. 


Controller 

Mean 

SD 

A 

4.11 

5.04 

B 

6.22 

2.82 

c 

4.00 

2.18 

Soldiers  were  able  to  successfully  perform  all  the  driving  and  reconnaissance  tasks  presented  on  the 
robotic  driving  course  using  all  three  controllers.  Upon  completion  of  each  iteration,  Soldiers  were 
asked  to  rate  the  characteristics  of  the  controller  they  had  used  and  their  ability  to  perform  various 
tasks  while  maneuvering  through  the  driving  course.  When  asked  to  rate  the  characteristics  of  each 
controller,  Soldiers  rated  controller  B  highest  for  size,  weight,  shape,  comfort  of  use,  and  location 
of  individual  controls.  Several  Soldiers  stated  that  controlling  the  robot’s  rate  of  speed  and 


19 


direction  using  the  B  controller  was  difficult  because  of  the  sensitivity  of  its  driving  controls. 
Soldiers  stated  that  the  large  size  of  controller  A’s  controls  allowed  them  to  be  labeled  and  made 
finding  the  correct  control  easy. 

Performing  multiple  tasks  simultaneously  (i.e.,  raising  the  control  arm  while  tilting  the  sensor 
head)  was  rated  as  very  difficult  with  the  C  controller.  Controlling  more  than  one  robotic  task 
simultaneously  with  one  hand  requires  more  dexterity  than  when  the  tasks  are  performed  with 
both  hands.  Several  Soldiers  reported  that  they  experienced  wrist,  arm,  hand,  and  finger  strain 
when  operating  the  C  controller.  Supporting  the  weight  of  the  weapon  while  manipulating  the 
controller  and  performing  all  tasks  with  one  hand  were  identified  as  the  primary  reasons  Soldiers 
were  experiencing  strain.  Detailed  Soldier  questionnaire  results  for  the  driving  course  are 
presented  in  appendix  A. 

3.5  NASA  TLX 

Table  13  shows  the  weighted  means  for  each  of  the  TLX  scales,  as  well  as  the  weighted  means 
for  the  total  workload  score.  The  scale  score  means  are  illustrated  in  figure  11.  The  graph 
clearly  indicates  that  the  mental  activities  formed  the  greatest  portion  of  task  workload.  The  C 
controller  has  a  slightly  higher  mean  workload  rating  than  controllers  A  or  B  in  terms  of  mental 
demands,  effort,  and  frustration. 


Table  13.  Weighted  means,  NASA  workload  scales. 


Controller 

Mental 

Physical 

Temporal 

Performance 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

A 

13.2 

9.2 

0.5 

0.9 

4.1 

2.6 

8.0 

7.2 

B 

13.3 

7.6 

1.2 

3.1 

6.2 

7.0 

6.8 

6.6 

c 

17.3 

9.1 

2.2 

2.9 

6.2 

6.2 

5.8 

5.3 

Controller 

Effort 

Frustration 

Total 

Mean 

SD 

Mean 

SD 

Mean 

SD 

A 

6.5 

5.3 

4.3 

5.3 

36.6 

13.5 

B 

5.3 

4.7 

7.0 

9.1 

39.7 

16.1 

c 

8.7 

5.8 

9.5 

10.6 

49.7 

23.8 
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Figure  11.  Mean  NASA  TLX  scale  scores. 

Repeated  measures  ANOVAs  were  used  to  compare  the  weighted  means  for  each  of  the  workload 
scales.  The  results  are  summarized  in  table  14.  The  difference  among  the  controller  means  was 
significant  only  for  the  total  scale. 


Table  14.  ANOVA  summary,  NASA  TLX  scales. 


Scale 

df 

F 

p 

»12p 

Mental 

2,28 

2.77 

0.080 

0.165 

Physical 

2,28 

3.24 

0.054 

0.188 

Temporal 

2,28 

1.27 

0.297 

0.083 

Effort 

2,28 

<  1.00 

0.638 

0.032 

Performance 

2,28 

1.92 

0.165 

0.121 

Frustration 

2,28 

1.97 

0.158 

0.123 

Total 

2,28 

3.93 

0.031* 

0.219 

*p  <  .05 


The  follow-on  pair-wise  comparisons  for  the  total  scale  are  summarized  in  table  15.  Although 
none  of  the  pair-wise  differences  were  statistically  significant,  the  difference  between  controllers 
A  and  C  approached  statistical  significance. 


Table  15.  Follow-on  comparisons,  total  NASA-TLX. 


Pair 

t 

df 

obtained  p 

required  p 

A  vs.  B 

-1.09 

14 

0.296 

0.05 

A  vs.  C 

-2.62 

14 

0.020 

0.0167 

B  vs.  C 

-1.62 

14 

0.127 

0.025 
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3.6  End  of  Experiment  Questionnaire  Results 

Most  Soldiers  felt  that  controller  A  was  the  quickest  and  easiest  to  use  because  of  its  size  and 
control  layout.  Controller  B  was  selected  by  most  Soldiers  as  the  most  comfortable  controller  to 
use  because  of  its  size,  weight,  and  design.  Controller  C  was  selected  as  the  easiest  to  carry  on  a 
mission  since  it  attaches  directly  to  the  weapon.  Soldiers  commented  that  they  would  like  the 
ability  to  set  the  sensitivity  level  for  each  controller,  based  on  the  difficulty  they  experienced 
controlling  the  robot’s  rate  of  speed  and  problems  encountered  with  over-steering  when 
maneuvering. 

Soldiers  felt  that  reducing  a  controller  size  by  shrinking  the  size  of  the  individual  controls  would 
impact  performance  less  than  reducing  its  size  by  reducing  the  number  of  controls.  Soldiers 
indicated  that  the  multifunction  controller  was  more  difficult  to  learn  and  use  than  the  controller 
with  reduced  control  sizes  and  that  switching  between  functions  was  time  consuming  and 
confusing.  Detailed  end  of  experiment  questionnaire  results  are  presented  in  appendix  A. 


4.  Conclusions 


The  primary  objective  of  this  study  was  to  evaluate  controller  devices  that  differ  in  features 
relevant  to  scaling  robotic  control  devices  for  the  dismounted  Soldier.  Soldiers  were  able  to 
successfully  perform  all  the  driving  and  reconnaissance  tasks  presented  on  the  robotic  driving 
course  using  all  three  controllers.  The  results  provide  an  indication  that  reducing  a  controller 
size  by  shrinking  the  size  of  the  individual  controls  (controller  B)  had  less  adverse  impact  on 
performance  than  reducing  its  size  by  reducing  the  number  of  controls  (controller  C).  However, 
when  one  is  interpreting  the  results  of  this  experiment,  it  is  important  to  understand  that  there  were 
limitations  in  the  availability  of  controllers  that  were  identical  in  all  aspects  (control  mappings, 
sensitivity,  etc.).  Thus,  uncontrolled  variables  could  have  played  a  role  in  the  outcomes.  However, 
several  findings  that  support  this  conclusion  should  not  have  been  impacted  by  the  uncontrolled 
variables.  First,  Soldiers  reported  that  the  multifunction  controller  was  more  difficult  to  learn  and 
use  than  the  controller  with  reduced  control  sizes  because  switching  between  functions  was  time 
consuming  and  confusing.  This  difficulty  increased  perceived  workload,  which  is  a  particularly 
important  finding  for  dismounted  operations  because  Sterling  and  Perala  (2007)  found  that  robotic 
controllers  supporting  infantry  units  had  higher  workloads  and  stress  than  controllers  supporting 
other  types  of  units.  Second,  Soldiers  found  that  several  robotic  control  functions  could  not  be 
performed  simultaneously  (e.g.,  raise  the  control  arm  while  turning  the  sensor  head)  with  the 
multifunction  controller.  This  necessitated  sequential  operation  which  was  time  consuming  and 
difficult.  Therefore,  despite  the  uncontrolled  variables,  the  conclusion  that  miniaturization  provides 
less  adverse  impact  on  performance  reduction  of  the  number  of  controls  appears  to  be  a  valid  one. 
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The  intuitiveness  test  used  during  this  experiment  to  evaluate  the  control  mapping  performed  on 
the  controllers  shows  promise  as  an  evaluation  technique.  It  would  be  especially  useful  as  a  quick 
and  easy  way  to  use  target  audience  Soldiers  to  evaluate  different  mappings  on  the  same  controller. 
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Appendix  A.  Soldier  Questionnaire  Results 


DEMOGRAPHICS 
SAMPLE  SIZE  =  15 


RANK 

MOS 

DUTY  POSITION 

AGE 

E-3  -  1 

1 1B  - 

1  68W 

-  2 

Rifleman 

-1 

28  years 

(mean) 

E-4  -  3 

13F  - 

1  92A 

-  1 

OCS 

-4 

E-5  -  6 

15H  - 

1  98G 

-  1 

Instructor  Pilot 

-1 

E-6  -  1 

25S  - 

1  OCS 

-2 

Medical  PSG 

-  1 

E-7  -  3 

25W  - 

1  NR 

-3 

S-1  Clerk 

-  1 

CW3-  1 

42-F  - 

1 

S-3  Clerk 

-1 

S-4 

-1 

NR 

-5 

1 .  How  long  have  you  served  in  the  military?  69  months  (mean) 

2.  How  long  have  you  had  an  infantry-related  job?  18  months  (mean) 

3.  How  long  have  you  been  a  fire  team  leader?  2  months  (mean) 

4.  How  long  have  you  been  a  squad  leader?  12  months  (mean) 

5.  How  long  have  you  been  deployed  overseas?  15  months  (mean) 

6.  How  long  have  you  been  deployed  in  a  combat  area?  7  months  (mean) 

7.  Corrected  Visual  Acuity:  14  20/20  both  eyes  0  20/20  in  one  eye  1  other:  20/60 

8.  Please  list  any  visual  problems  which  you  may  have.  None  (other  than  corrective  lenses). 

9.  Height:  69  inches  (range  is  60-76) 

10.  Weight:  169  pounds  (range  is  143-210) 

1 1 .  With  which  hand  do  you  most  often  write?  13  Right  2  Left 

12.  With  which  hand  do  you  most  often  fire  a  weapon  ?  15  Right  0  Left 

13.  a.  Do  you  wear  prescription  lenses?  9  No  6  Yes 

b.  If  so,  which?  4  Glasses  2  Contact 

Lenses 

14.  Do  you  wear  glasses  or  contacts  while  firing  a  9  No  6  Yes 

weapon? 

15.  Which  is  your  dominant  eye?  1 1  Right  4  Left 

16.  Have  you  ever  driven  a  robotic  vehicle?  If  yes,  15  No  0  Yes 

describe. 

17.  Have  you  ever  driven  a  remote  control  car?  4  No  9  Yes  2  NR 
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18.  Using  the  scale  below,  please  rate  your  level  of  experience  with  the  following  computer 
software  and  computer  related  activities. 


1 

2 

3 

4 

5 

6 

7 

No 

experience 

Below 

average 

Slightly 

below 

average 

Average 

Slightly  above 
average 

Above 

average 

Expert 

MEAN  RESPONSE 

a.  Microsoft  Windows  98,  2000,  XP,  etc. 

4.80 

b.  Computer  based  games 

4.27 

c.  Army  digital  systems  (e.g.  FBCB2) 

2.53 

d.  1  would  self  rate  my  computer  skills  as: 

4.73 

19.  Using  the  scale  below,  please  rate  the  following  Self-rating  of  Knowledge,  Skills,  and 
Abilities  (KSAs)  related  to  Infantry  duties. 


1 

2 

3 

4 

5 

6 

7 

No 

experience 

Below 

average 

Slightly 

below 

average 

Average 

Slightly  above 
average 

Above 

average 

Expert 

MEAN  RESPONSE 

a.  Knowledge  of  tactics,  techniques,  and  procedures  (TTP). 

3.87 

b.  Knowledge  of  map  reading  and  orientation  in  field  setting. 

4.53 

c.  Knowledge  of  reconnaissance,  surveillance,  and  target 
acquisition  procedures. 

2.87 

d.  Knowledge  relating  to  communications  equipment  and 
communications  procedures. 

3.73 

e.  Communication  skills  (ability  to  use  communications  equipment 
and  face-to-face  communications  to  enhance  mission 
accomplishment). 

4.27 

20.  How  often  do  you  play  video  games? 

3  Never  5  1-1 1  times  per  year  3  1-3  times  per  month  2  1-6  times  per  week  2 
Daily 

21 .  List  the  games  you  play  regularly  from  1  (most  frequent)  to  5  (least  frequent). 

1.  Counter  strike,  John  Madden,  Halo  (2),  Medieval  Total  War,  Rise  of  Nations,  Tetris 

2.  Day  of  Defeat,  NCAA  Football,  Project  Gotham  Racing  3,  WarCraft,  Halo, 
Command/Conquer2,  Gran  Trurismo 

3.  Halo,  NCAA  Basketball,  Sudoku,  Age  of  Empires  II,  Solitaire,  Command/Conquer  Generals, 
Grand  Theft  Auto 

4.  MLB  Baseball,  Brain  Age,  Morrowind,  Dominoes,  Oblivion 

5.  NHL  Hockey,  Everquest,  Chess,  Lego  Starwares  2 
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POST  ITERATION 


SAMPLE  SIZE  =  15 

1 .  Using  the  scale  below,  please  rate  the  training  that  you  received  on  the  controller  in  the 
following  areas: 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


TRAINING  TOPICS 

MEAN  RESPO 

NSE 

A 

B 

C 

Explanation  of  how  to  drive  the  robot 

6.53 

6.47 

6.20 

Time  provided  to  practice  driving  the  robot 

6.47 

6.40 

5.93 

Explanation  of  how  to  operate  the  controller 

6.67 

6.67 

6.13 

Time  provided  for  explanation  of  the  controller 

6.67 

6.53 

6.07 

Difficulty  of  learning  the  controller 

6.40 

5.67 

4.40 

Explanation  of  how  to  complete  each  station  of  the  course 

6.87 

6.53 

6.67 

Evaluation  of  the  practice  lane 

6.67 

6.47 

6.33 

How  well  did  you  expect  to  perform  on  the  actual  course  after 
your  training  was  complete? 

5.33 

5.27 

5.20 

Overall  evaluation  of  the  training  course 

6.27 

6.20 

5.20 

2.  What  were  the  easiest  and  hardest  training  tasks  to  learn? 

Comments  No.  of  Responses 

Easiest: 


A 

Overall  control.  1 

Learning  how  to  operate.  1 

What  each  control  did.  1 

Climbing  the  stairs  and  hill.  1 

Driving.  3 

Steering.  1 

Using  the  direction  controller.  1 

Using  the  controller.  2 

Easy  layout.  1 

Camera  control.  1 

Camera  movement.  1 

Moving  the  robot.  1 

Speed  control.  1 
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Comments 


No.  of  Responses 


Multi-tasking.  1 

Using  the  selection  screen.  1 

B 

Self-explanatory.  1 

Learning  the  controller.  1 

Buttons  easy  to  remember.  3 

Forward  movement.  1 

Movement  and  maneuver.  1 

Utilizing  camera  and  display.  1 

Flippers.  1 

Controlling  head.  1 

Control  of  robot.  1 

Directional  control.  1 

Controls  very  well  organized.  1 

Stationary  task  (view  stationary  targets).  1 

Use  of  control  sticks  for  movement.  1 

C 

Learning  the  controller.  1 

Driving.  2 

Driving  over  the  hill.  1 

Taking  corners.  1 

Steering  the  control.  2 

Automatic  drive  mode.  1 

Moving  the  body.  1 

Thumb  stick.  2 

Camera.  2 

Learning  what  each  button  does.  1 

Nothing  was  easy.  1 

Hardest: 

A 

Following  and  focusing  on  moving  targets.  1 

Learning  how  the  orientation  icon  was  oriented.  Sometimes  it  was  facing  me,  1 

sometimes  away. 

Performing  each  step.  1 

Adapting  to  the  display’s  lag.  1 

Movement  (turning).  1 

Neck  movement.  1 

Flipper  and  neck  control.  1 

The  maze.  1 

Tracking  eye  movement.  1 
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Comments 


No.  of  Responses 


Camera  maneuver.  1 

Azimuth.  1 

Using  zoom.  1 

T racking  targets.  1 

The  scrolling  of  the  camera  on  moving  object.  1 

B 

Steering.  1 

Controlling  the  speed.  2 

Sensitivity.  4 

Compensating  for  lag  time  on  display.  2 

Camera.  1 

Camera  angle.  1 

Control  turning.  1 

All  the  buttons.  1 

No  Y  axis  inversion.  1 

Limited  tracking  ability.  1 

Tracking  a  moving  person.  1 

The  cross  of  the  flipper  control  and  the  raising  and  lowering  of  arm.  The  flipper  1 

control  should  be  on  the  side  of  the  drive  and  the  arm  control  should  be  on  the 
side  of  the  camera. 

C 

Backing  up  the  robot  into  a  spot  where  the  camera  has  the  most  visibility.  1 

Controller  hard  to  move.  1 

Camera  modes.  1 

Camera  switching.  1 

Controlling  the  camera.  1 

Camera  toggle  (select).  2 

Narrow  zigzag  hallway.  1 

Holding  on  to  the  controller.  1 

Toggling  between  different  control  functions.  1 

Learning  and  switching  between  modes.  2 

Tracking  moving  targets.  1 

GUI.  1 

3.  What  are  your  comments  on  the  training  course? 

A 

Excellent.  1 

Very  simple  to  understand  and  learn.  2 

Very  well  set-up  and  built  course.  3 

Suitable  for  real  life  scenarios.  1 


31 


Comments 


No.  of  Responses 


Adequate  difficulty.  1 

Maybe  incorporate  more  use  of  the  camera  on  moving  objects.  1 

Put  the  robot  through  good  range  of  maneuvers.  1 

Misc.  tasks  allow  for  different  aspects  of  the  robot  being  able  to  be  used  and  1 

learned. 

Could  use  more  obstacles.  1 

The  lag  between  control  and  actual  is  very  disorienting  though  with  time  a  person  1 
can  adjust. 

B 

Very  good.  1 

Lots  of  fun!  1 

Objectives  were  clear,  instructions  kept  simple,  and  tasks  useful  for  real  life  1 

situations. 

Good  course;  covered  most,  if  not  all,  areas  that  robot  would  be  used  in.  2 

Consistent  and  enables  you  to  use  all  the  features.  1 

Good  maneuvers  for  the  robot  and  used  all  controls.  1 

Testing  my  ability  on  basic  tasks.  1 

Much  easier  after  one  training  period.  1 

Needs  a  little  more  practice  (operator)  before  testing.  1 

Joystick  was  too  sensitive  to  movement.  1 

Incorporate  more  camera  movement  tasks.  1 

C 

Overall  good  instruction.  1 

Very  good  course.  4 

Very  realistic  on  everyday  combat  fields.  1 

Very  easy  to  get  a  working  knowledge  of  this  system.  1 

Good  for  learning  controls.  1 

Used  a  variety  of  tasks  on  the  robot.  1 

Evaluated  basic  tasks;  taught  very  well.  1 

Could  be  more  challenging.  1 

The  control  was  fairly  simple,  but  having  it  on  the  weapon  was  not  practical.  1 

Need  a  little  more  training  time  in  the  foxhole.  1 

The  course  was  more  difficult  and  extremely  time  consuming.  1 

Controller  was  slippery  and  failed  to  go  R  and  L.  1 

More  tracking  of  moving  targets.  1 

4.  What  did  you  think  about  this  controller  after  completing  the  training  course? 

A 

Great  controller.  1 

I  liked  how  the  controller  was  easy  to  function  and  easy  to  learn  and  operate.  3 

Practical  setup.  1 

Everything  is  at  your  fingertips.  1 

Very  easy  to  use.  6 
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Comments 


No.  of  Responses 


Easy  to  multi-task  with.  1 

Sensitive.  1 

Overall,  good  controller.  1 

Good  control.  2 

Well  labeled.  1 

Excellent,  but  I  had  azimuth  issues.  1 

Interface  was  good  and  the  buttons  were  under  utilized.  If  some  of  the  buttons  1 
were  used  instead  of  generic  arrows/select,  it  could  be  more  efficient,  i.e.,  zoom 
in/out. 

Adequate.  1 

Big,  bulky.  3 

Needs  a  few  improvements.  1 

It  works,  yet  the  program  seemed  too  overbearing  on  the  computer.  1 

Control  sticks  could  be  more  sensitive.  1 

Could  not  multifunction  enough,  i.e.,  flippers  and  driving.  1 

B 

Easy  and  quick  to  use.  2 

Easiest  to  use  by  far.  1 

Good  size.  1 

Easy  to  reach  all  controls.  1 

Very  simple;  almost  like  common  gaming  controls.  1 

The  controller  was  familiar,  making  the  learning  curve  a  little  easier.  1 

Key  placement  was  intuitive,  making  learning  simple.  2 

I  liked  the  ergonomics  of  it  a  lot.  Makes  more  sense  than  the  other  two.  1 

It  was  OK.  1 

I  like  it,  but  I  would  change  the  drive  button.  1 

Better  fine  control  (but  not  inherent  in  the  controller,  more  in  calibration).  1 

Limited  multifunction.  1 

Why  waste  a  whole  button  on  brake?  Could  be  used  as  a  default/home  button.  1 
For  example,  I  want  to  go  to  drive  mode  with  drive  camera  from  anywhere. 
Acceleration  issues  with  thumb  force  controller.  1 

Way  too  sensitive.  5 

Having  the  zoom  button  and  the  toggle  to  “track”  ,both  on  the  right  side,  made  it  1 
difficult  to  do  both  tasks  simultaneously. 

Tactically,  wouldn’t  have  positive  control  of  your  weapon.  1 

Too  many  unlabeled  buttons.  The  number  sequence  made  it  more  difficult  to  1 

remember  what  each  was  meant  for. 

Hard  for  me  to  control  camera  (L,  R,  up,  down).  1 

C 

Easy  to  use.  2 

Very  easy  to  learn  and  move.  1 

By  far  the  best  maneuvering  control.  1 

Very  smooth.  1 
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Comments 


No.  of  Responses 


Practical.  1 

Ergonomic.  Don’t  change  a  thing  with  the  controller.  1 

Multifunction  button  with  computer  screen  makes  it  simple  to  navigate.  1 

Button  placement  is  easy  to  work  with.  1 

The  one  positive  is  the  system  being  attached  to  the  weapon,  but  the  equipment  1 
has  a  greater  chance  of  getting  damaged. 

It’s  better  as  just  a  hand-held  device.  1 

Very  complicated.  1 

Need  some  grips  on  the  controller  so  your  hand  won’t  slip.  1 

I  can’t  see  the  thumb  stick  working  well  when  the  operator  is  sweaty.  I  kept  slipping  1 
off. 

Takes  too  much  time  to  switch  between  modes.  If  I  was  in  a  hurry,  it  would  have  1 
been  hard  to  execute. 

Camera  controller  should  be  inverted.  1 

May  be  some  steering  bugs.  1 

Did  not  like.  1 

Not  very  good.  1 

Sticky  and  doesn’t  offer  sensitive  controls.  1 

Speed  is  not  consistent.  1 

This  controller  is  terrible,  unreliable,  and  difficult  to  maneuver.  1 


5.  Using  the  scale  below,  please  rate  your  ability  to  perform  each  of  the  following  control  tasks 
based  on  your  experience  with  the  controller  that  you  just  used. 


1  2  3  4  5  6  7 

Extremely  Very  difficult  Difficult  Neutral  Easy  Very  easy  Extremely 
difficult  easy 


CONTROL  TASKS 

MEAN  RESPO 

NSE 

A 

B 

C 

Ascend  stairs 

5.60 

5.80 

5.40 

Descend  stairs 

5.93 

5.80 

5.60 

Position  robot  to  look  through  a  window 

5.67 

5.13 

4.33 

Scan  for  targets 

5.40 

5.33 

4.40 

T rack  a  target 

4.60 

4.40 

4.53 

Zoom  in  and  out  with  the  camera 

5.73 

5.87 

4.53 

Maintain  control  over  uneven  terrain 

5.93 

5.20 

5.27 

Position  robot  to  look  in  a  bunker 

5.60 

5.53 

4.20 

Drive  through  a  tunnel 

6.33 

6.20 

5.43 

Drive  through  a  confined  space  (zigzag) 

5.07 

4.80 

4.73 

Stay  within  course  limits  (engineer  tape) 

5.13 

4.53 

5.20 

Control  robots  rate  of  speed 

4.87 

3.40 

4.40 

Raise  robot  arm 

6.40 

6.27 

4.86 

Tilt  sensor  head 

6.00 

5.73 

4.87 

(continued  on  next  page) 
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(continued) 


CONTROL  TASKS 

MEAN  RESPO 

NSE 

A 

B 

C 

Rotate/pan  sensor  head 

6.20 

5.93 

4.93 

Steer  robot  while  moving  forward 

4.73 

4.13 

4.80 

Steer  robot  while  moving  in  reverse 

4.64 

4.36 

4.43 

Position  robot  flippers 

6.40 

6.40 

5.00 

Perform  multiple  tasks  simultaneously  (i.e.  raise  arm  while 
tilting  sensor  head) 

5.14 

4.71 

2.54 

Overall  ability  to  perform  driving  tasks 

5.87 

5.13 

4.87 

Comments  No.  of  Responses 


A 

Excellent.  1 

Easy  to  operate.  2 

Much  easier  to  use.  1 

User  friendly.  1 

Great  control  on  speed.  1 

Easy  to  do  the  task.  I  could  drive,  raise  the  head,  and  change  cameras  all  together.  1 
This  controller  made  minute  changes  possible  and  easy.  1 

Size  is  too  bulky.  1 

Due  to  the  lag  time,  there  was  difficulty  adjusting  to  the  drift.  1 

Difficult  to  make  slight  turns.  2 

I  had  some  difficulty  with  overcorrecting  on  right  and  left  steering.  1 

Never  understood  how  to  “tilt”  the  camera  head.  The  view  ended  up  uneven  when  1 
trying  to  track  a  target.  Difficulty  keeping  it  level. 

Hardest  part  was  trying  to  do  multiple  tasks,  i.e.,  tracking  targets  while  tilting  head 
up  and  down. 

The  falling  of  the  flippers  and  having  to  raise  them  over  and  over  again  while  1 

traveling  over  flat  ground  detracted  from  concentration  on  the  movement. 

B 

Very  responsive  to  the  controls,  but  lag  made  it  easy  to  overcompensate.  1 

Difficult  to  steer  while  moving  forward,  otherwise  it  was  good.  1 

Hard  to  adjust  drifting  without  overcompensating.  1 

Turning  right  tends  to  pull  backward.  1 

Steering  functions  were  impaired  due  to  the  sensitivity  of  the  analog  sticks.  1 

Speed  control  was  an  issue  for  me.  The  controller  is  very  sensitive.  3 

Seemed  like  it  moved  like  a  button  instead  of  a  positional  switch.  1 

To  control  the  robot  with  this  specific  controller  was  more  difficult  and  took  more  1 

time  to  maneuver. 

Moving  tasks  are  much  more  difficult  than  stationary.  With  practice,  task  can  1 

become  easier  to  perform. 


35 


Comments 


No.  of  Responses 


One  control  stick  should  control  the  forward  movement  the  other  should  control  left  1 


and  right  movement.  Also,  movement  should  be  controlled  by  right  hand. 

C 

Relatively  easy  to  work  with  limited  exposure  to  device.  1 

Much  easier  to  use  for  me.  1 

Driving  and  turning  at  the  same  time  was  very  easy.  1 

Sensitivity  of  thumb  stick  was  good  because  it  helped  get  rid  of  over  correction.  1 

Not  being  able  to  do  multiple  tasks  at  the  same  time.  3 

Not  being  able  to  perform  multiple  tasks,  as  with  zoom  in  or  out  while  tracking  1 

made  it  difficult  to  pinpoint  what  an  object  is.  More  concentration  was  needed  to 
switch  to  the  modes  making  the  task  at  hand  more  time  consuming. 

Nearly  impossible  to  perform  tasks  simultaneously,  but  it  would  not  take  long  to  1 
learn  to  perform  functions  quickly. 

Having  to  toggle  between  different  stations  is  really  frustrating.  2 

The  need  to  switch  modes  made  me  unable  to  zoom  out  while  tracking  my  target.  1 
I  think  the  flippers  and  sensor  head  should  switch  positions  on  the  controller.  1 

One  or  two  more  hours  using  the  controller  would’ve  made  a  difference.  1 

The  camera  toggle  sticks  or  jumps.  1 


6.  During  this  trial,  did  you  experience  any  of  the  following? 


Number  of  Responses 

A 

B 

c 

Wrist  strain 

0 

0 

1 

Arm  strain 

0 

0 

4 

Hand  strain 

0 

0 

5 

Finger  Strain 

0 

0 

4 

Nausea 

0 

0 

0 

Fatigue 

0 

0 

0 

Disorientation 

0 

1 

0 

Dizziness 

0 

0 

0 

Other? 

0 

0 

0 

C 

Fingers  slipping.  1 

Got  tired  from  arm  and  hand  strain.  1 

Wrist  strain;  soreness/CT.  1 

Arm  strain  holding  the  weapon.  1 

Arm  got  tired.  1 

Hand  strain  having  to  use  fingertips.  2 

Finger  strain  using  toggle  with  thumb.  2 

Could  be  physically  straining  if  had  to  perform  for  long  periods  of  time.  1 
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7.  Using  the  scale  below,  please  rate  the  following  characteristics  of  the  controller  that  you  just 
used: 


1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


CONTROLLER  CHARACTERISTICS 

MEAN  RESPO 

NSE 

A 

B 

C 

Size  of  controller 

4.00 

6.33 

5.00 

Size  of  individual  controls 

4.93 

6.33 

5.07 

Location  of  individual  controls 

5.67 

6.07 

4.67 

Comfort  while  holding  the  controller 

4.64 

6.40 

4.53 

Weight  of  controller 

4.07 

6.40 

4.80 

Shape  of  controller 

4.33 

6.27 

4.87 

Mapping  of  controller  functions 

5.60 

5.87 

3.67 

8.  Using  the  scale  below,  what  is  your  overall  rating  of  the  controller  that  you  used  this  iteration? 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN  RESPONSE 

A 

B 

C 

5.36 

5.67 

4.50 

Comments  No.  of  Responses 


A 

Positive.  1 

Good  controller.  1 

Easy  to  use  and  learn.  2 

User  friendly.  1 

Controllability  is  good.  1 

Very  good  control  over  speed  of  vehicle.  1 

Operates  very  smooth.  1 

Very  well  labeled  and  layout  was  ideal.  1 

Good  sensitivity  on  camera  adjustments.  1 

A  large  controller  offers  more  functions  and  ease  of  use.  In  a  situation  where  size  1 
didn’t  matter,  this  would  be  a  great  controller. 

If  I  had  to  use  as  hand-held,  instead  of  placing  on  a  table,  I’m  sure  it  would  be  1 

extremely  difficult  to  maneuver. 

Negatives  outweighs  the  positives.  1 

Make  it  more  ergonomic  to  allow  for  ease  of  operation  while  holding.  1 

Should  weigh  more.  1 
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Comments 


No.  of  Responses 


A  bit  large.  1 

Needs  to  be  a  little  more  sensitive  with  the  driving  controls.  1 

B 

The  size,  feel,  and  location  of  buttons  is  awesome.  1 

Easy  to  learn.  1 

Very  light;  fits  comfortably  in  my  hand.  1 

Shape,  weight,  familiarity  of  design  was  good.  1 

Control  buttons  ideally  placed.  1 

I  like  it;  very  ergonomic.  1 

I  liked  it  as  it  was  very  responsive.  1 

It  would  help  if  the  controller  could  be  configured  to  the  operator,  i.e.,  right-handed  1 
person  drives  with  right  toggle. 

Takes  getting  used  to  for  the  time  lag.  1 

Not  best  for  tactical  environment,  but  otherwise  outstanding.  1 

Complicated.  1 

The  mapping  of  the  button  functions  needs  work.  1 

C 

Very  easy  to  use.  1 

I  can’t  think  of  a  better  place  on  an  M4  or  in  a  combat  environment  to  have  this  1 
controller  than  where  it  is  now. 

The  thumb  stick  should  have  a  depression  in  it  for  your  thumb.  1 

Controller  needs  a  textured  thumb  controller  to  avoid  slipping  in  a  very  hot  1 

environment. 

Less  sensitive  than  B  and  allows  for  more  precise  movements  that  are  easily  1 

controlled. 

Cycling  through  a  main  menu  can  be  complicated.  1 

Requires  too  much  time  overall.  1 

Would  like  to  see  multiple  task  options.  1 

Just  too  hard  to  multitask.  You  need  to  be  able  to  easily  transition  between  modes.  1 
Maybe  eliminate  some. 

Difficult  and  frustrating  to  keep  switching  between  functions  to  make  it  perform.  1 
Don’t  like  one-hand  control.  1 
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END  OF  EXPERIMENT 


SAMPLE  SIZE  =  15 

1 .  Using  the  scale  below,  please  rank  order  the  controllers  in  the  order  of  your  preference  on  the 
following  characteristics  by  placing  a  “1”  next  to  your  favorite,  a  “2”  next  to  your  second  choice,  etc. 

1  2  3 

First  choice  Second  choice  Last  choice 


CHARACTERISTICS 

MEAN  RESPO 

NSE 

A 

B 

C 

a.  Comfort  in  the  hand 

2.33 

1.33 

2.33 

b.  Ease  of  use 

1.60 

1.73 

2.67 

c.  Quickest  to  use 

1.53 

1.67 

2.80 

d.  Ease  of  carrying  with  you  on  a  mission 

2.80 

1.73 

1.47 

e.  Best  to  take  with  you  in  battle 

2.47 

1.73 

1.80 

Comments 


No.  of  Responses 


Comfort  in  the  hand 
A 

Most  rugged. 

Better  controllability. 

Still  solid  with  both  hands  working. 

A  familiar  shape,  but  awkward. 

Too  large;  bulky. 

If  smaller,  it  would  be  better. 

Very  difficult  to  hold. 

B 

Fit  well  in  my  hands. 

Hand-held  controller. 

Very  comfortable. 

Lightest. 

Had  better  ergonomics  and  access  to  buttons. 

Very  natural. 

Most  practical.  Easily  used  while  holding. 

Easier  to  function. 

Mapped  ideally  and  clearly  labeled. 

Allows  multi-functions  using  thumbs  and  fingers. 

It  feels  like  a  game  controller. 

Too  sensitive. 

Hard  to  control  speed. 


1 

2 

1 

1 

3 

1 

1 

2 

1 

1 

1 

2 

1 

1 

1 

1 

1 

1 

2 

1 


39 


Comments 


No.  of  Responses 


C 

It  fits  in  palm  of  one  hand.  2 

Easier  for  me  to  control  rate  of  speed  and  azimuth  simultaneously.  1 

Ergonomic  and  easy  to  use.  1 

Awkward,  but  convenient.  1 

Very  tricky  to  change  commands.  1 

Very  difficult.  1 

Too  complex  for  simple  tasks.  1 

Limited  to  one  hand  which  can  experience  fatigue.  1 

Cumbersome.  1 

Too  much  with  the  weapon.  1 

Ease  of  use 
A 

Very  basic  and  large,  making  it  easy  to  use.  5 

Simple.  2 

Controls  are  very  practical.  1 

Less  sensitive  so  lag  wasn’t  difficult  to  adjust.  1 

Targeting  is  rather  easy  on  all.  1 

Less  buttons  and  all  were  labeled.  2 

Not  as  well  mapped/labeled  as  B.  1 

Too  bulky.  2 

Joysticks  multi-functioning  a  poor  idea.  1 

Almost  everything  is  done  with  two  controls  on  this  one.  1 

B 

Game  controller.  1 

Easy  to  use.  3 

Clearly  marked;  no  confusion;  natural.  1 

Every  button  could  be  reached  easily.  Your  hands  never  had  to  leave  the  1 

controller. 

Light  and  ready  to  use  with  functions  at  fingertips.  1 

Small,  but  more  buttons.  1 

Little  to  no  switching  between  modes.  1 

A  close  second.  1 

Targeting  is  rather  easy  on  all.  1 

Very  difficult  to  use.  1 

C 

Easier  for  me  to  control  rate  of  speed  and  azimuth  simultaneously.  1 

Targeting  is  rather  easy  on  all.  1 

4  buttons.  1 

Didn’t  function  right.  1 

Very  difficult  to  use.  1 

Display  system  much  more  complicated  than  others;  time  consuming.  1 

Not  as  fluid.  1 
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Comments 


No.  of  Responses 


Difficult  to  find  the  control  you  need.  1 

A  lot  of  hit  and  miss  to  change  modes.  1 

Takes  a  few  minutes  to  get  used  to.  1 

Quickest  to  use 
A 

Every  use  written  on  keypad/controller.  2 

Big,  labeled  buttons.  2 

Very  natural.  1 

No  multi-tasking.  1 

I  could  have  easily  figured  out  the  controls  without  any  guidance.  1 

All  controls  could  be  reached  from  one  position,  never  requiring  much  movement.  1 
B 

More  single  function  buttons.  2 

Closest  to  the  controllers  I’m  used  to,  so  it  was  the  easiest  to  just  pick  up  and  use.  1 
Most  familiar  to  me.  1 

Most  sensitive.  1 

Limited  multi-tasking.  1 

Too  many  buttons  for  functions.  1 

Required  some  movement.  1 

Set  up  much  like  a  gaming  controller.  1 

C 

All  multi-function  buttons. 

Modes  are  a  work  around  for  limited  inputs,  but  having  more  inputs  is  essential  for  1 
multi-function. 

Controllers  were  easy  to  use.  I  could  toggle  through  and  select  tasks.  1 

Very  difficult  to  switch  modes.  1 

Had  too  much  mode  switching.  1 

Slow  because  you  have  to  cycle  thru  everything.  1 

Required  almost  constant  movement  to  change  modes.  1 

Nothing  is  labeled  and  takes  a  little  getting  used  too.  1 

Ease  of  carrying  with  you  on  a  mission 

This  is  a  size  issue.  1 

A 

Fit  as  a  natural  piece  of  the  weapon.  1 

Smallest/lightest.  1 

Too  big,  bulky.  3 

B 

Lightest.  4 

Easy  and  small.  5 

Small,  but  still  is  extra  component.  1 

C 

Just  attaches  to  your  weapon  and  you  don’t  have  to  think  about  it.  4 
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Comments 


No.  of  Responses 


Device  C  if  it  was  not  attached  to  the  weapon.  1 

Would  be  right  on  the  rifle.  1 

Very  practical  since  it  may  attach  to  the  RAS.  1 

Too  large.  2 

Obvious  here.  1 

Size  of  controller  matters  for  packing  in  ruck.  1 

Will  work  on  an  M-4,  but  at  what  cost  to  the  warrior?  1 

Best  to  take  with  you  in  battle 

This  is  a  size  issue.  1 

A 

Fit  as  a  natural  piece  of  the  weapon.  1 

Smallest,  lightest.  1 

Easy  to  use.  1 

Actually  works.  1 

Too  big,  bulky.  2 

B 

Second  best  since  it  is  so  small.  3 

Durable.  1 

Light.  2 

Easiest  to  use  without  looking  once  functions  are  learned.  1 

Small,  but  still  is  extra  component.  2 

C 

Best  since  it  is  on  your  weapon.  1 

Already  on  weapon  and  ready  to  use.  1 

Attaches  to  the  weapon  and  you  can  maintain  a  hand  on  the  grip  of  the  weapon.  1 
But  since  you  are  so  involved  in  the  operation  of  the  robot,  you  won’t  be  able  to 
effectively  use  your  weapon. 

Convenient,  but  in  a  bad  place  for  battery  to  break  off  or  get  damaged.  1 

Too  large.  2 

Looks  like  PAQ-4C  and  front  M-4  handle.  1 

Will  work  on  an  M-4,  but  at  what  cost  to  the  warrior?  1 

Compacting  equipment  is  always  practical.  1 

This  would  not  be  something  a  rifleman  should  have.  1 

2.  Please  provide  suggestions  for  improvement  in  the  following  areas: 

a.  Suggestions  for  ways  to  increase  the  effectiveness  of  the  controllers. 

A 

Smaller  design.  3 

Easier  to  handle.  1 

Use  the  buttons  for  multi-function  tasks.  1 
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Comments 


No.  of  Responses 


Lets  users  map  their  own  controls.  1 

No  malfunction  with  joysticks.  1 

Be  able  to  select  the  sensitivity  of  the  thumb  sticks!!!  1 

If  it  is  a  two-handed  controller,  make  it  where  it  can  be  tailored  to  the  person,  i.e.,  1 

left  or  right-handed. 

Decrease  lag.  1 

Hand-held  separate  controller.  1 

Less  sensitivity  of  controllers.  2 

Zoom  control  on  the  twist  axis  of  the  controllers.  1 

B 

Attach  to  weapon  somehow.  1 

Have  both  thumb  sticks  on  the  control  movement.  1 

Make  the  two-handed  controller  where  you  can  adjust  the  sensitivity.  1 

Make  just  a  hair  more  rugged.  1 

Make  more  user  friendly.  1 

Use  face  buttons  for  3  separate  cameras  and  last  for  select.  1 

Less  switching  between  modes  the  better.  1 

Reduce  sensitivity  of  steering.  1 

Minimize  camera  delay.  1 

Remove  superfluous  buttons.  1 

Wireless.  1 

C 

Small  joystick  on  it.  1 

Easier.  1 

Faster  to  use.  1 

When  zoomed  in,  sensitivity  goes  down  so  you  don’t  over  shoot.  1 

Make  the  one-handed  controller  able  to  be  multi-functioning.  1 

Make  less  sensitive  to  overcome  lag.  1 

Input  into  a  HUD.  1 

Create  a  function  to  rotate  180  degrees.  1 

Smaller.  1 

A  textured  grip.  1 

Able  to  adjust  rate  of  acceleration.  1 

Not  really  a  good  idea.  1 

Move  brake  to  another  place  (or  make  it  automatic).  1 

Use  bottom  button  for  drive  mode  default  (a  home  button).  1 

Remap  buttons;  awkward  to  select  with  pinky  and  ring  finger..  1 

Add  buttons;  using  display  navigation  is  slow.  1 

Tighten  the  toggle  for  less  exaggerating  of  maneuvers.  1 
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Comments 


No.  of  Responses 


b.  Suggestions  on  ways  to  improve  the  training  course. 

Course  was  great.  3 

Did  a  great  job  of  simulating  the  needs  of  the  equipment.  1 

Shorter  day.  1 

Multi-testing  by  all  testers.  1 

More  tasks  that  require  multi-tasking.  1 

More  tracking  of  targets.  1 

Track  a  moving  target  over  uneven  terrain.  1 

T rack  a  target  while  moving.  1 

Going  under  obstacles,  like  cars.  1 

Raise  the  engineer  tape  to  six  inches.  1 

Controls  and  different  maneuvers.  1 

Make  course  harder.  1 

More  obstacles.  2 

Add  IED  or  other  training  robot.  1 

Reduce  training  time  to  30,  then  25,  then,  20.  1 

Vary  course  a  little.  Can  be  repetitive.  Go  thru  the  other  way.  1 

Multiple  robots  at  once.  1 

Use  tactical  situations.  1 

Increase  difficulty.  1 

c.  Suggestions  on  ways  to  improve  the  driving  lanes  that  you  negotiated 

No  need  to  change;  accomplished.  1 

Desired  mission.  1 

Rooms  (urban  terrain).  1 

Make  one  longer  to  see  how  people  adjust  for  slight  drifting  in  forward  movement.  1 
Make  driving  lane  more  defined.  1 

Make  lanes  rougher  to  negotiate.  3 

Maneuver  more  turns.  1 

Drive  with  lights  on  (dark).  1 

More  terrain  obstacles  (i.e.,  wet  ground,  mud,  rubble).  3 

Longer  straight-aways.  1 

Make  some  with  overhead  cover.  1 

d.  Which  would  you  prefer? 

4  One-handed  control  operation 

11  Two-handed  control  operation 
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Comments 


No.  of  Responses 


One-handed 

Allows  free  use  of  other  hand  for  tactical  environments,  et. 
Easier. 

Needs  to  be  multi-taskable. 


2 

1 

1 


Two-handed 

Allows  for  more  ease  of  control  and  makes  for  an  overall  less  complicated  system.  1 
Keeping  maneuvers  to  the  left  and  specific  tasks  on  the  right  worked  well.  1 

To  change  modes.  1 

Means  I  can  do  more  at  one  time.  1 

Feels  more  stable.  1 

Requires  less  mode  changes  which  can  complicate/slow  down  process.  1 

Opens  up  more  options  for  functions.  1 


e.  Did  you  have  any  difficulty  operating  the  controls  on  the  Q3D  with  your  left  hand? 

4  Yes 
11  No 


No.  In  fact,  I  used  my  left  hand,  but  only  because  I  felt  like  I  had  to  have  my  firing  1 
hand  on  my  weapon. 

Didn’t  require  dominant-arm-strength/coordination.  Controls  were  simple  enough  to  1 


use  a  non-dominant  hand. 

1 

I’m  used  to  firing  my  weapon  left-handed  so  it  made  it  easier.  3 

Just  changing  menus  was  challenging.  1 

Too  much  switching  between  modes.  1 

I  would  have  to  have  a  lot  of  practice  with  this  device.  1 

Got  fatigued  quickly.  1 

Very  uncomfortable.  1 


3.  Using  the  scale  below,  please  rank  the  following  features  in  terms  of  importance  in  a  robotic 
hand  controller  by  placing  a  “1”  next  to  your  favorite,  a  “2”  next  to  your  second  choice,  etc. 


12  3  4 

First  choice  Second  choice  Third  choice  Fourth  choice 


5 

Fifth  choice 


6 

Last  choice 


MEAN 

RESPONSE 

Comfort  in  hand 

2.79 

Size 

3.07 

Weight 

4.21 

Ruggedness 

4.57 

Maintenance  free 

3.57 

Intuitiveness  of  controls 

2.60 
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4.  Please  answer  the  following  questions  on  the  entire  controller  system  (processor,  antennae, 
cables,  etc.) 

a.  Would  you  want  to  carry/wear  the  controller  system? 

10  Yes 

_ 4_  No 

_ 1_  Don’t  know 

Comments  No.  of  Responses 

No,  Solders  carry  enough  already.  2 

It’s  a  laptop  and  controller.  It  would  weigh  less  than  the  radios.  1 

Why  would  we  wear  it?  Keep  it  in  a  case  with  the  robot.  1 

Did  not  experience  the  weight  of  the  robot  itself.  If  light  enough,  this  piece  of  1 

equipment  will  serve  a  great  purpose  to  any  Solder. 

Too  fragile  for  combat.  1 

b.  Please  provide  as  many  adjectives  as  you  can  think  of  describing  only  the  hardware  of  the 
controller  system  (processor,  antennae,  cables,  display,  hand  controller,  etc.) 

Expensive.  1 

Durable.  1 

Easy.  2 

Basic.  1 

Small.  2 

Compact.  1 

Lightweight.  1 

Efficient.  1 

Laggy.  1 

Rugged.  2 

Rough.  1 

Cool.  1 

Agile.  1 

Versatile.  1 

Impressive  picture.  1 

Programming  bugs.  1 

Heavy.  1 

Need  to  be  condensed.  1 

Too  many  wires  hanging  out.  1 

Not  too  rugged  looking.  1 

A  bit  much  with  a  full  combat  load  with  body  armor.  1 

Simple,  yet  unreliable.  2 

c.  If  you  could  name  the  robot  you’ve  been  controlling,  what  would  you  name  it? 

Threat  Negator.  1 

Stalker.  1 

Bob.  1 

Johnny  5.  4 

Besides  Johnny  5?  How  about  Chuck.  I’ve  never  known  a  Chuck,  but  this  robot  1 

sure  acts  like  one. 

Roger.  1 
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Comments 


No.  of  Responses 


Urban  surveillance  electronic  robot  (USER).  1 

Recon  ranger.  1 

The  Mole.  1 

Breland-Bot.  1 

Red  Max;  Red  Hot.  1 


6.  Using  the  scale  below,  please  rate  the  importance  of  the  following  driving  tasks  that  are  needed 
for  teleoperating  a  robot. 

1  2  3  4  5  6  7 

Extremely  Very  Very  Extremely 

Unimportant  Unimportant  Unimportant  Neutral  Important  Important  Important 


DRIVING  TASKS 

MEAN  RESPONSE 

a.  Heading  information 

5.20 

b.  Relative  distance  to  obstacles 

5.93 

c.  Depth  of  pot  holes 

5.67 

d.  Information  on  the  navigability  of  down  slopes  (tilt  meter) 

5.53 

e.  Information  on  the  navigability  of  side  slopes  (tilt  meter) 

5.40 

f.  Identification  of  other  terrain  features  that  might  have  an  adverse 
effect  on  the  ability  of  the  robot  to  maneuver  through  the  terrain 
(describe) 

5.43 

g.  Ability  to  determine  where  the  sides  of  the  vehicle  are  located, 
(sides  are  in  the  camera  field  of  view  or  are  marked  on  the  screen) 

5.60 

h.  Ability  to  determine  where  the  front  of  the  vehicle  is  located 

5.73 

i.  Ability  to  determine  where  the  back  of  the  vehicle  is  located 

4.93 

j.  Information  concerning  whether  the  ground  clearance  of  the 
vehicle  will  k.  allow  negotiation  of  rugged  terrain 

5.27 

1.  Indication  of  the  turn  radius  of  the  vehicle 

5.00 

m.  Information  on  the  terrain  closer  than  6  inches  in  front  of  the 
vehicle 

5.57 

n.  Information  on  the  terrain  five  to  15  feet  in  front  of  the  vehicle 

5.13 

o.  Information  on  the  terrain  greater  than  15  feet  in  front  of  the 
vehicle 

4.67 

p.  Information  on  the  current  speed  of  the  vehicle  (speedometer) 

4.20 

q.  Information  on  vehicle  RPMs 

3.27 

r.  Information  on  the  way  the  vehicle  motor  sounds 

4.40 

s.  Information  on  the  other  noises  present  in  the  vehicle’s 
environment 

5.13 

t.  Information  concerning  the  color  of  objects  in  the  environment 
(color  vs.  black  and  white  camera) 

5.20 

u.  Information  concerning  objects  on  the  side  of  the  vehicle  (side 
facing  cameras) 

5.00 

v.  Information  concerning  objects  behind  the  vehicle  (back  facing 
camera) 

5.13 

w.  Information  concerning  the  condition  of  the  vehicle  tires/tracks. 

5.00 

(continued  on  next  page) 
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(cont) 


DRIVING  TASKS 

MEAN  RESPONSE 

x.  Information  concerning  the  temperature  of  the  vehicle 

4.60 

y.  Feedback  on  the  ruggedness  of  terrain. 

4.67 

z.  Information  concerning  any  delay  between  the  time  the  operator 
sends  a  vehicle  command  and  the  time  that  the  vehicle  responds 
(i.e.,  how  long  after  a  stop  command  is  sent  before  the  vehicle 
actually  stops) 

5.73 

aa.  Feedback  concerning  whether  or  not  the  vehicle  correctly 
received  a  command 

4.87 

bb.  Battery  status 

6.20 

cc.  Fuel  status 

6.07 

dd.  Vehicle  temperature 

5.27 

Comments  No.  of  Responses 

f.  sand/muck,  rocket,  clearance  under  objects.  1 

Most  important  is  lag  time  info.  1 

I  think  360  degree  awareness  around  the  robot  is  important  because  someone  1 

could  easily  sneak  up  on  or  sneak  by  otherwise.  Also,  many  threats  could  go 
undetected  without  360  degree  awareness. 

You  should  know  how  loud  the  robot  is  being  if  it  must  be  stealthy.  1 

Color  is  important  because  it  can  go  to  identification.  1 

The  lag  could  pose  challenges  until  it  is  the  user  adjusts  to  it.  The  B  controller  1 

would  be  the  best  option  regarding  size  and  mapping  of  controls. 

A  HUD.  1 


7.  The  purpose  of  this  experiment  was  to  determine  the  feasibility  of  reducing  the  size  of  the 
robotic  controller  for  use  by  dismounted  Solders.  Based  upon  what  you’ve  seen,  is  it  feasible  to 
reduce  the  size  of  the  controller  without  adversely  impacting  performance  by: 

a.  Shrinking  the  controller  by  shrinking  it’s  controls  sizes? 

10  Yes 
5  No 


Shrink  Device  A.  1 

Device  B  was  easiest  to  use  and  had  the  smallest  learning  curve.  1 

Device  C  is  a  very  good  size.  1 

Device  C  is  perfect  for  combat.  1 

Shrinking  controls  seems  relatively  uncomplicated.  1 

Not  worth  shrinking  anymore;  all  three  are  fairly  small.  1 

The  smaller,  the  more  comfortable  it  was  to  use.  1 

Smaller  controls  don’t  impact  the  performance.  1 

The  sizes  are  correct  for  the  size  of  the  individual  controls.  1 

If  having  to  carry  it,  it  would  be  useful  if  smaller.  1 
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b.  Shrinking  the  size  of  the  controller  by  reducing  the  number  of  controls? 

5  Yes 
10  No 


Comments  No.  of  Responses 

For  the  first  two.  1 

Will  increase  time  spent  performing  control  tasks.  1 

It  is  easier  to  control  the  robot  and  can  more  timely  do  so  with  more  buttons.  1 

If  done  right,  this  could  work,  but  this  prototype  wasn’t  user  friendly.  1 

Make  more  user  friendly.  1 

To  reduce  much  would  adversely  affect  performance.  1 

Having  to  switch  between  modes  to  complete  a  maneuver  wasted  time.  1 

More  controls  are  needed  to  make  it  faster  and  easier.  1 

Device  C  swapped  user  friendliness  for  less  controls  -  bad!  1 

Having  the  press  the  same  button  over  and  over  was  annoying.  1 
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Intentionally  left  blank 
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Appendix  B.  Control  Intuitiveness  Test  Results 


Controller  A 

1.  Raise  the  arm  of  the  sensor  head. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1A 

2 

2 

1 

5 

11 

IB 

1 

0 

1 

2 

4 

1C 

0 

0 

1 

1 

1 

IE 

0 

0 

1 

1 

1 

2 

3 

2 

0 

5 

13 

3 

0 

0 

1 

1 

1 

5 

1 

1 

4 

6 

9 

6 

0 

2 

0 

2 

4 

11 

0 

0 

1 

1 

1 

14 

1 

6  | 

1 

2 

4 

18A 

3 

2 

0 

5 

13 

18B 

1 

2 

0 

3 

7 

18E* 

0 

1 

0 

1 

2* 

18F 

0 

0 

1 

1 

1 

2.  Raise  the  flippers. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1A 

2 

1 

0 

3 

8 

IB 

0 

0 

1 

1 

1 

1C 

0 

1 

0 

1 

2 
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(cont) 


2.  Raise  the  flippers. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

IE 

1 

0 

0 

1 

3 

IF* 

1 

0 

0 

1 

3* 

2 

3 

0 

2 

5 

11 

4 

0 

0 

1 

1 

1 

5 

0 

2 

1 

3 

5 

10 

0 

0 

1 

1 

1 

11 

1 

0 

0 

1 

3 

12 

0 

0 

1 

1 

1 

13 

0 

1 

0 

1 

2 

14 

0 

2 

1 

3 

5 

17 

2 

1 

2 

5 

10 

18A 

0 

2 

0 

2 

4 

18B 

0 

1 

1 

2 

3 

18F 

2 

0 

0 

2 

8 

3.  Drive  the  robot  forward. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1A* 

9 

5 

0 

14 

37* 

IB 

0 

1 

0 

1 

2 

18A 

3 

0 

1 

4 

10 

4.  Brake. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

IB 

2 

1 

0 

3 

7 

1C 

0 

0 

1 

1 

1 

IE 

0 

0 

1 

1 

1 

1G 

1 

0 

0 

1 

3 

2 

0 

1 

0 

1 

2 

3 

0 

0 

1 

1 

1 

4 

0 

0 

1 

1 

1 

5 

1 

0 

0 

1 

3 

14* 

2 

2 

2 

6 

12* 

15 

2 

1 

0 

3 

8 

16 

0 

2 

1 

3 

5 

17 

3 

0 

1 

2 

10 

18B 

1 

2 

1 

4 

8 

18G 

0 

1 

0 

1 

2 

5.  Pan  the  camera  left. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1C 

2 

2 

1 

5 

11 

IE 

3 

0 

0 

3 

9 
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(cont) 


5.  Pan  the  camera  left. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

6 

0 

1 

1 

2 

3 

10 

1 

1 

0 

2 

5 

11 

0 

0 

1 

1 

1 

14 

1 

0 

0 

1 

3 

18C* 

4 

2 

1 

7 

17* 

18E 

1 

2 

1 

4 

8 

‘Correct  choice  based  on  current  mapping. 

Controller  A 


1.  Raise  the  Arm 


First  Choice 

Response  Reason  selected  No.  Responses 


1A  Direction;  on  left  1 

Guess  1 

IB  Intuitive;  flight  analogy  1 

2  Important  function;  should  have  easy  access  1 

Intuitive  1 

5  Guess  1 

14  Button  is  easy  to  reach  and  prefer  to  use  joystick  1 

18A  Assume  left  toggle  for  driving;  camera  second 

most  important  1 

Main  function,  so  should  be  main  control  1 

Video  game  analogy  1 

18B  Seems  most  logical;  left  stick  for  driving,  right  for 

camera  1 

Second  Choice 
Response 


1A  Directional  motion 

Easy  to  operate 


1 

1 
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No.  of  Responses 


2  Each  row  could  represent  a  different  part  of  the 

robot  1 

Guess  1 

5  Guess  1 

6  Process  of  elimination  1 

Would  want  close  to  left  hand  1 

18A  Guess  1 

Joystick  seems  probable  1 

18B  Guess  1 

Second  most  logical  1 

18E  Process  of  elimination;  joysticks  appear  for  track  so 

must  be  button  1 

Third  Choice 
Response 

1A  Most  sensitive  control  1 

IB  Joystick  natural  for  controlling  direction  1 

1C  Left  or  right  motion  makes  sense  for  moving  head  1 

IE  Guess  1 

3  Top  of  the  key  pad  equals  up  1 

5  Closest  button  if  driving  with  right  hand  1 

Easy  access  1 

Guess  1 

11  Could  be  arrow  pad  1 

14  Guess  1 

18F  Simple  movement  1 

2.  Raise  the  Flipper 

First  Choice 
Response 

1A  Move  quickly  1 

Up  and  down  motion  1 
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No.  of  Responses 


11 

Two  directional  analog 

1 

IE 

Turn  for  up  or  down 

1 

2 

Each  row  for  different  part  of  robot 

1 

Guess 

1 

Top  of  control 

1 

11 

Buttons  may  be  arrow  pad 

1 

17 

Easy  to  reach 

1 

Flippers  located  opposite  of  neck  on  robot  so 
opposite  on  keyboard 

1 

18F 

Guess 

1 

Joystick  for  movement 

1 

Second  Choice 

Response 

1A 

Assigns  direction 

1 

1C 

Same  as  1 ;  hands  already  on  stick  for  driving 

1 

5 

Guess 

1 

13 

Process  of  elimination 

1 

14 

Easy  to  reach  with  non  driving  hand 

1 

17 

Guess 

1 

18B 

Direction  easy  to  control 

1 

Third  Choice 

Response 

IB 

Guess 

1 

2 

Guess 

2 

4 

Process  of  elimination 

1 
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No.  of  Responses 


5 

Assume  would  be  upper  button 

1 

10 

Same;  process  of  elimination 

1 

12 

Process  of  elimination 

1 

14 

Easy  to  reach  with  thumb 

1 

17 

Easy  to  reach  with  driving  hand 

Guess 

1 

18B 

Could  use  it  while  driving 

1 

Drive  the  robot  forward. 

First  Choice 

Response 

1A 

Directional 

1 

Feels  natural 

1 

Makes  sense;  easy  to  use 

2 

Left  hand  dominant 

1 

Most  sensitive  to  direction;  assume  left 

1 

Usually  left  controls  body 

1 

Guess 

1 

Two  track  should  have  two  movements 

1 

18A 

Drive  with  right  hand 

1 

Steering  easier  with  right  hand 

2 

Second  Choice 

Response 

1A 

Guess 

2 

Next  best  choice 

1 

Process  of  elimination 

1 

Second  most  logical 

1 

IB 

Same  as  1  but  opposite  direction 

1 

Third  Choice 

Response 

18A 

If  not.  two  controls  would  drive  with  right  hand 

1 
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4.  Brake. 


First  Choice 
Response 

No.  of  Responses 


IB  Could  easily  stop  1 

Opposite  of  driving  forward  1 

1G  Brake  analogy  1 

5  Won’t  use  often  so  furthest  away  1 

14  Easy  to  reach  quickly  1 

Makes  sense  1 

15  Guess  2 

17  Brake  should  be  near  drive  control  1 

If  driving  with  right  hand  easiest  to  reach  1 

Intuitive  to  use  thumb  for  brake  1 

18B  Opposite  of  driving  forward  1 


Second  Choice 
Response 


IB 

Opposite  direction  of  motion 

1 

2 

Same  as  first 

1 

14 

Easiest  to  reach  with  other  hand 

1 

Same  as  1 

1 

15 

Close  to  drive;  lower  end  of  keypad 

1 

16 

Guess 

2 

18B 

Same  as  1 

2 

18G 

Same  as  1 

1 
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Third  Choice 
Response 


No.  of  Responses 


1C  Wouldn’t  want  to  push  button  1 

IE  Twisting  motion  like  pan  motion  1 

3  Guess  1 

4  Guess  1 

14  Easily  reachable  from  drive  control  1 

Same  as  1  1 

16  Process  of  elimination  1 

17  Peripheral  1 

18B  Pull  back  to  stop  1 

5.  Pan  the  camera  left. 


First  Choice 
Response 

1C  If  driving  with  right  hand  would  want  camera  on  the 


left  1 

Left  side  1 

IE  Guess  1 

Knob  has  left  and  right  direction  1 

Turn  left  with  left  hand  1 

10  Hadn’t  used  row  yet  1 

14  Left  side  of  keyboard;  lower  priority  function  1 

18C  Guess  1 

If  related  to  moving  the  neck  should  also  move  the 
camera  1 

Intuitive  1 

Sensitive  control  1 

18E  Same  as  1  1 
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Second  Choice 
Response 


No.  of  Responses 


1C 

First  person  shooter  game  analogy 

1 

Left  side 

1 

6 

Second  most  intuitive 

1 

10 

Process  of  elimination 

1 

18C 

Feels  most  natural 

1 

Next  best  choice 

1 

18E 

Guess 

1 

Same  as  1 

1 

Third  Choice 

Response 

1C 

Guess 

1 

6 

Guess 

1 

11 

Process  of  elimination 

1 

18C 

Easier 

1 

18E 

If  not  for  driving  makes  sense  for  camera  control 

1 

59 


Controller  B 


Controller  B 

1.  Raise  the  arm  of  the  sensor  head. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1 

0 

1 

0 

1 

2 

2* 

2 

2 

2 

6 

12* 

3 

3 

1 

3 

7 

14 

5 

0 

1 

0 

1 

2 

9A 

1 

1 

1 

3 

6 

9B 

1 

0 

1 

2 

4 

10A 

4 

2 

0 

6 

16 

10B 

1 

0 

1 

2 

4 

16 

0 

2 

2 

4 

6 

17 

2 

1 

1 

4 

9 

18 

1 

1 

0 

2 

5 
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2.  Raise  the  flippers. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1 

2 

0 

0 

2 

6 

2 

3 

0 

1 

4 

10 

3 

1 

2 

1 

4 

8 

7 

0 

0 

1 

1 

1 

9A 

1 

1 

1 

3 

6 

9D 

0 

0 

1 

1 

1 

10A 

1 

2 

0 

3 

7 

12 

0 

1 

1 

2 

3 

16 

2 

1 

1 

4 

9 

17* 

5 

3 

1 

9 

22* 

18 

0 

1 

0 

1 

2 

3.  Drive  the  robot  forward. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

2 

0 

0 

1 

1 

1 

3 

4 

2 

1 

7 

17 

9A* 

8 

5 

2 

15 

36* 

9B 

0 

1 

0 

1 

2 

10A 

2 

1 

1 

4 

9 

16 

0 

1 

0 

1 

2 

4.  Brake. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1 

0 

1 

0 

1 

2 

IB 

1 

0 

0 

1 

3 

2 

1 

0 

1 

2 

4 

5 

1 

0 

0 

1 

3 

7* 

1 

0 

2 

3 

5* 

8 

0 

2 

1 

3 

5 

9B 

1 

1 

0 

2 

5 

9E 

0 

0 

1 

1 

1 

10E 

1 

1 

0 

2 

5 

11 

3 

0 

0 

3 

9 

12 

0 

1 

1 

2 

3 

13 

2 

1 

1 

4 

9 

14 

3 

3 

3 

9 

18 

15 

1 

1 

0 

2 

5 

17 

0 

2 

1 

3 

5 

18 

0 

1 

3 

4 

5 
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5.  Pan  the  camera  to  the  left. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1 

1 

0 

0 

1 

3 

2 

0 

1 

3 

4 

5 

3 

0 

1 

0 

1 

2 

4 

4 

2 

2 

8 

18 

9C 

4 

3 

1 

8 

19 

IOC* 

4 

3 

3 

10 

21* 

10D 

1 

0 

0 

1 

3 

‘Current  correct  choice. 


Controller  B 


1.  Raise  the  Arm 


First  Choice 

Response  Reason  selected  No.  Responses 


2  Guess  1 

Makes  sense  1 

3  Axis  of  movement  appears  similar  to  robot  1 

Guess  1 

Other  direction  controls  assumed  to  be  for  other 

tasks  1 

9A  Directional  control  1 

9B  Prefer  to  drive  with  right  and  use  left  for  peripheral  1 

10A  Assume  left  toggle  for  driving;  camera  second 

most  important  1 

Easy  to  reach  and  move  1 

Pivot  control;  right  handed  1 

Right  handed  1 

10B  Seemed  related  to  arm  motion  1 


17  Digital  analog  of  up  and  down  1 

Makes  sense  1 

18  Easy  to  reach  1 
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Second  Choice 
Response 

No.  of  Responses 

1  Same  as  1  1 

2  Can  press  quickly  1 

Same  as  1  1 

3  Arrow  up  makes  sense  to  move  up;  video  game 

analogy  1 

5  Next  closest  to  left  hand;  pull  up  analogy  1 

9A  Direction  1 

Same  sort  of  control  as  first  choice  1 

10A  More  comfortable;  directional  1 

T ry  different  direction  1 

16  13-16  directional  mapping  1 

Right  side  preference  1 

17  Right  handed  1 

18  Opposite  orientation  of  first  choice  1 

Third  Choice 
Response 

2  Feels  like  up  1 

Same  as  1  1 

3  Arrow  signifies  up  1 

Direction  arrow  2 

9A  Same  as  2  1 

9B  Seems  to  be  toggle  1 

10B  Only  logical  choice  left  1 

16  Four  button  could  represent  axis  of  movement  1 

Top  equals  up  1 

17  Same  as  1  1 
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1 


Head  top  feet  bottom 
Convenient  for  up  and  down 


1 

1 


2 

Looks  like  physical  layout  of  the  robot 

1 

Seems  logical  for  up 

1 

Flippers  independent 

1 

9A 

One  control  for  each  flipper 

1 

10A 

Right  handed 

1 

16 

Neck  control  and  flipper  control  should  be  parallel 

1 

Up  direction 

1 

17 

Guess 

Lower  priority  task;  out  of  the  way;  don’t  need  high 

2 

sensitivity 

1 

Makes  sense 

1 

Seem  to  relate  to  up  and  down 

1 

Second  Choice 

Response 

3 

Direction 

1 

Same  as  1 

1 

9A 

Direction  makes  sense 

1 

10A 

Drive  with  right  hand 

1 

Process  of  elimination 

1 

12 

Guess 

1 

16 

Up  direction 

1 

17 

If  left  buttons  for  neck  then  right  for  flippers 

1 

Same  as  1 

2 

18 

Same  as  1 

1 
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Third  Choice 
Response 

No.  of  Responses 


2  Top  and  bottom  correspond  to  up  and  down;  one 

set  for  each  flipper  1 

3  Up  arrow  1 

7  Guess  1 

9A  Next  logical  choice  1 

9D  Two  flippers  should  have  two  controls  1 

Looks  like  it  would  be  up  and  the  one  below  it  down  1 

Makes  sense  1 

Top  button;  quick  raise  1 

3.  Drive  the  robot  forward. 


First  Choice 
Response 


3  Direction  arrow  1 

Direction  pad  typical  for  forward  movement;  video 
game  analogy  1 

Forward  arrow  1 

Video  game  analogy  1 

9A  Easier  to  maneuver  1 

Easy  to  use  with  thumb;  main  function  1 

Easy  to  use;  video  game  function  1 

Has  swivel  for  direction;  more  freedom  of 
movement  1 

Most  sensitive  to  direction  1 

Control  speed  and  direction  1 

Direction;  on  for  each  track  1 

Tank  analogy  1 

10A  Drive  with  right  hand  1 

Driving  most  important  task;  right  handed;  sensitive 
to  pressure,  gas  pressure  analogy  1 

Right  handed;  analog  stick  1 
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Second  Choice 
Response 


No.  of  Responses 


3 

Forward  arrow 

1 

Only  other  logical  choice 

1 

9A 

Next  logical  choice 

1 

Process  of  elimination;  left  hand  dominant 

1 

Same  as  1 

1 

Video  game  analogy 

1 

One  control  for  each  track 

1 

9B 

Same  as  1  but  opposite  direction 

1 

10A 

Right  handed;  seems  logical 

1 

16 

Right  handed 

1 

Third  Choice 

Response 

2 

Process  of  elimination 

1 

3 

Directional  pad 

1 

9A 

Easy  to  use  analog  stick 

1 

Process  of  elimination 

1 

10A 

Guess 

1 

4.  Brake- 

First  Choice 
Response 


IB  Can  slow  while  driving  1 

2  Close  to  forward  button  1 

5  Pull  back  brake  analogy  1 

7  Process  of  elimination  1 

9B  Opposite  of  drive  forward  1 
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No.  of  Responses 

10E  Driving  control;  brake  analogy  1 

11  Closest  to  driving  control  1 

Low  priority  function;  don’t  want  to  accidentally  1 

press;  video  game  pause  button 
Process  of  elimination  1 

13  Analogy  to  video  game  1 

Video  game  analogy  1 

14  Closest  to  thumb;  wouldn’t  be  directional  1 

Should  be  a  button  type  control;  close  to  thumb  is  2 

convenient 

15  Should  be  a  button  type  control  1 

Second  Choice 
Response 

1  Hands  off  all  other  buttons  1 

8  Easy  to  reach;  doesn’t  seem  like  priority  1 

Same  as  1  1 

9B  Backward  motion  to  stop  1 

10E  Intuitive  1 

12  Easy  to  reach  with  thumb  1 

13  Same  as  1  1 

14  Drive  with  left  and  brake  with  right  1 

Easy  to  reach  with  thumb  1 

Feels  natural  to  press  for  brake  1 

15  Next  to  first  choice  1 

17  Out  of  the  way  of  other  controls  1 

Same  as  1 ;  proximity  to  index  finger  makes  in  1 

convenient 

18  Guess  1 
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Third  Choice 
Response 

No.  of  Responses 


2 

Easy  to  press  quickly 

1 

7 

Not  easily  confused  with  other  controls 

1 

Same  as  2 

1 

8 

Guess 

1 

9E 

Makes  sense 

1 

12 

Same  as  1 

1 

13 

Guess 

1 

14 

Down  position 

1 

Process  of  elimination 

1 

Seems  unrelated  to  other  functions  so  off  to  the 
side 

1 

17 

Guess 

1 

18 

Easy  access 

1 

Easy  to  reach  with  fingers 

1 

Same  as  1 ;  process  of  elimination 

1 

5.  Pan  the  camera  left. 

First  Choice 
Response 


1  Guess  1 

4  Directional  pad  2 

Easy  to  change  1 

Makes  sense  1 

9C  Easy  to  move  with  toggle  control  1 

Joystick  should  move  camera  left  and  right  and  the  1 

neck  up  and  down 

Non  dominant  hand;  high  priority  function;  speed  1 

and  direction 

Right  to  drive;  left  for  camera;  video  game  analogy  1 
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No.  of  Responses 

IOC  Affordance  to  turn  360  degrees;  right  handed  1 

If  left  toggle  is  drive  the  right  should  be  for  the  1 

camera 

Logical  choice;  right  control  usually  for  head  1 

Sensitive  control  1 

Point  in  direction  of  motion  1 

13  Buttons  13-16  create  up-down,  left  -right  layout  1 

Second  Choice 
Response 

2  Left  control;  top  button  can  be  pressed  quickly  1 

3  Left  arrow  1 

4  Directional  arrows  1 

Directional  pad  1 

9C  Left  side  and  left  direction  1 

Makes  sense  1 

Next  to  first  choice  1 

10C  Directional;  drive  with  left  hand,  control  camera  with  1 

right 

If  not  for  driving  would  make  sense  1 

Prefer  to  use  left  hand  for  view  1 

13  Wouldn’t  use  toggle  1 

Third  Choice 
Response 

2  Could  use  simultaneously  with  other  functions  1 

Easy  access;  on  left  side  1 

Guess  1 

4  Direction  arrow  1 

Only  directional  control  left  1 

9C  Guess  1 

10C  Process  of  elimination  1 

Same  as  1  1 

Toggle  direction  1 
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Controller  C 


Controller  C 

1.  Raise  the  arm  of  the  sensor  head. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1A 

8 

4 

2 

14 

34 

IB 

0 

2 

0 

2 

4 

ID* 

0 

0 

1 

1 

2* 

2 

4 

5 

0 

9 

22 

3 

1 

1 

5 

7 

10 

4 

0 

0 

3 

3 

3 

5 

0 

0 

1 

1 

1 

6A 

0 

1 

0 

1 

2 

2.  Raise  the  flippers. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1A* 

4 

3 

3 

10 

21* 

2 

3 

3 

0 

6 

15 
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(cont) 


2.  Raise  the  flippers. 

3 

1 

2 

5 

8 

12 

4 

3 

1 

2 

6 

13 

5 

1 

1 

0 

2 

5 

6A 

1 

0 

0 

1 

3 

6B 

0 

1 

0 

1 

2 

3.  Drive  the  robot  forward. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1A* 

13 

0 

0 

13 

39* 

4.  Brake. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

IB 

1 

0 

1 

2 

4 

IE 

4 

1 

1 

6 

15 

2 

2 

4 

0 

6 

14 

3 

0 

0 

3 

3 

3 

4 

1 

0 

0 

1 

3 

5* 

4 

3 

0 

7 

18* 

6A 

0 

0 

1 

1 

1 

6B 

1 

0 

0 

1 

3 

7 

0 

1 

0 

1 

2 

5.  Pan  the  camera  to  the  left. 

Control 

First  Choice 

Second 

Choice 

Third 

Choice 

Total 

Weighted 

Totals 

1C* 

10 

5 

3 

18 

43* 

2 

0 

1 

0 

1 

2 

4 

1 

0 

0 

1 

3 

5 

0 

0 

1 

1 

1 

6B 

1 

0 

0 

1 

3 

‘Current  correct  choice. 


Controller  C 


1.  Raise  the  Arm 


First  Choice 

Response  Reason  selected  No.  Responses 


1A  Convenient  1 

Directional  pad;  multipurpose  2 

If  in  correct  mode  obvious  direction  1 

Only  control  with  direction  1 

Makes  sense  1 
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No.  of  Responses 

Since  only  one  direction  control  choose  a  button  to 
specify  mode  1 

Buttons  for  different  modes  1 

2  Easiest  to  reach  1 

Primary  function;  top  equals  up  2 

Starting  from  the  top  and  work  down  1 

3  Meant  to  hit  top  button;  up  goes  with  top  button  1 

Second  Choice 
Response 

1A  Directional  motion  1 

Logical  1 

Place  in  mode  1 

Guess  1 

IB  Convenient  1 

Only  directional  control  1 

2  If  not  toggle,  should  be  top  button  to  equal  up  1 

Main  function  so  first  button  1 

Top  button  means  raise  1 

Toward  top;  more  priority  1 

Up  goes  with  higher  located  button  1 

3  Process  of  elimination  1 

6A  Standard  position  of  neck  is  down  so  turn  grip  to 

the  right  to  raise  1 

Third  Choice 
Response 

1A  Process  of  elimination  1 

Guess  1 

ID  Right  and  left  could  mean  up  and  down  1 

3  Assume  the  first  two  buttons  correspond  to  head  1 

Guess  2 

Next  in  line;  primary  function  1 

Process  of  elimination  1 
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No.  of  Responses 

4  Buttons  are  two  pairs  of  up  and  down  1 

Process  of  elimination  2 

5  Guess  1 

2.  Raise  the  Flipper 

First  Choice 
Response 

1A  Direction  1 

Directional  control  1 

Logical  1 

Flippers  lower  priority  1 

2  Lower  relative  to  other  buttons  1 

Secondary  priority  so  not  the  toggle  1 

Top  button  equals  up  1 

3  Guess  1 

4  Feels  right  1 

Flippers  lower  priority  1 

Guess  1 

5  Flippers  lower  priority  1 

6A  Think  toggle  is  for  driving,  so  process  of  elimination  1 

Second  Choice 
Response 

1A  Direction  1 

Guess  2 

2  Guess  1 

Convenient  1 

Top  equals  up  1 

3  Guess  1 

Middle  location  on  robot  1 

4  Same  as  1 ;  process  of  elimination  1 

5  Order  of  priority;  process  of  elimination  1 


73 


6B 


Same  as  1 


No.  of  Responses 

1 


Third  Choice 
Response 


1A  If  not  for  driving;  directional  control  1 

Guess  1 

Mode  1 

3  Guess  4 

Same  as  1  1 

4  Guess  2 


3.  Drive  the  robot  forward. 

First  Choice 
Response 

1A  Directional  control 

Driving  most  important  task 
Feels  natural 

Flexible  to  control  movement 

Main  function 

Most  sensitive  to  direction 

Movement  is  natural  for  thumb;  up  equals  forward 
Only  one  that  makes  sense 
Range  of  motion 


3^ 

None. 

3^ 

None. 

4.  Brake. 

First  Choice 

Response 


IB  More  control  to  move  with  thumb  1 

IE  Brake  analogy  1 

Easy  if  driving  1 

Guess  1 

Makes  sense;  standard  in  other  controls  1 

2  Guess  1 

Closest  to  driving  control  1 


5 

1 

1 

1 

1 

1 

1 

1 

1 
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No.  of  Responses 

4  Most  convenient  1 

5  Furthest  away  prevents  confusion  1 

Least  frequent  use  so  furthest  away  1 

Low  priority;  out  of  the  way  1 

Out  of  the  way  1 

6B  Feels  like  halt,  pull  back  1 

Second  Choice 
Response 

IE  Brake  in  car  analogy  1 

2  Guess  1 

Close  to  drive  1 

Easy  access  1 

Makes  sense  to  press  a  button  for  brake  1 

5  Guess  2 

Very  end;  low  priority  1 

7  If  stop  pressing  controls  should  stop  robot  1 

Third  Choice 
Response 

IB  Guess  1 

IE  Guess  1 

3  Guess  1 

Process  of  elimination  2 

6A  Easy  to  reach  1 

5.  Pan  the  camera  left. 

First  Choice 
Response 

1C  Directional  4 

Left  direction  1 

More  flexible  than  buttons  1 

Toggle  is  for  direction  1 
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No.  of  Responses 


Guess  1 

Next  main  function  1 

Feels  comfortable  1 

4  Third  button  should  be  left  and  fourth  right  for 

camera  1 

6B  Left  handed  1 

Logical  1 


Second  Choice 
Response 


1C  Makes  sense  to  use  toggle  to  control  view  direction  1 

More  access  1 

Easy  1 

Guess  2 

2  Same  logic  as  one  for  first  and  second  button  1 

Third  Choice 
Response 

1C  Guess  2 

Next  in  line  1 

5  Guess  1 
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FT  LEONARD  WOOD  MO  65473 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MD  T  COOK 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 

1  COMMANDANT  USAADASCH 
ATTN  ATSA  CD 

ATTN  AMSRD  ARL  HR  ME  DR  HAWLEY 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 


NO.  OF 

COPIES  ORGANIZATION 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MM  DR  V  J  RICE 
BLDG  4011  RM  217 
1750  GREELEY  RD 
FT  SAM  HOUSTON  TX  78234-5002 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MG  R  SPINE 
BUILDING  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARL  HRED  ARMC  FLD  ELMT 

ATTN  AMSRD  ARL  HR  MH  C  BURNS 
BLDG  1467B  ROOM  336 
THIRD  AVENUE 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 
AWC  FIELD  ELEMENT 
ATTN  AMSRD  ARL  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BEL  VOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
91012  STATION  AVE  ROOM  348 
FT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MY  M  BARNES 
2520  HEALY  AVE  STE  1172  BLDG  51005 
FT  HUACHUCA  AZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MP  D  UNGV ARSKY 
POPE  HALL  BLDG  470  BCBL 
806  HARRISON  DR 
FT  LEAVENWORTH  KS  66027-2302 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MJF  J  HANSBERGER 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PARKWAY  SUITE  B 
SUFFOLK  VA  23435 
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1 
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ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MQ  M  R  FLETCHER 
US  ARMY  SBCCOM  NATICK  SOLDIER  CTR 
AMSRD  NSC  WS  E  BLDG  3  RM  343 
NATICK  MA  01760-5020 

ARMY  RSCH  LABORATORY-HRED 
ATTN  AMSRD  ARL  HR  MT  J  CHEN 
12423  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

ARMY  RSCH  LABORATORY-HRED 
ATTN  AMSRD  ARL  HR  MT  C  KORTENHAUS 
12350  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MS  C  MANASCO 
SIGNAL  TOWERS  BLDG  29808A  RM  303 
FORT  GORDON  GA  30905-5233 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MU  M  SINGAPORE 
6501  E  1 1  MILE  RD  MAIL  STOP  284 
BLDG  200 A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MF  C  HERNANDEZ 
2421  NW  AUSTIN  RD  STE  220 
FORT  SILL  OK  73503-9042 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 

ARMY  G1 

ATTN  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  ROOM  2C489 

WASHINGTON  DC  20310-0300 

US  ARMY  RDECOM  TARDEC 

ATTN  AMSRD  TAR  R  MS  264  TIERNEY 

WARREN  MI  48397-5000 


1  SOLDIER  BATTLE  LAB 

VIRTUAL/CONSTRUCTIVE  SIM  DIV 
ATTN  J  WILLIAMS 
6651  WAY  STREET 
FT  BENNING  GA  31905 

1  UNIV  OF  CENTRAL  FLORIDA 
ATTN  PHYC  DEPT  P  HANCOCK 
4000  CENTRAL  FL  BLVD 
ORLANDO  FL  32816 

3  SHAWN  STAFFORD 

2584  CREEKVIEW  CIRCLE 
OVIEDO  FL  32765 

1  JAMES  MERLO 

1047  KELLY  CREEK  CIRCLE 
OVIEDO  FL  32765 

1  iROBOT  CORPORATION 
ATTN  MELISSA  LEDOUX 
8  CROSBY  DRIVE 
BEDFORD  MA  01730 

ABERDEEN  PROVING  GROUND 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  Cl  OK  TECH  LIB 
BLDG  4600 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  Cl  OK  TP  S  FOPPIANO 

BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  HR  MR  F  PARAGALLO 

BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  HR  SC  K  COSENZO 
BLDG  459 
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